Tuesday, March 12, 2013

Hadoop overwrite options

There's an undocumented feature (Hadoop's documentation needs some serious love) that allows you to overwrite the destination just like you can with Unix's cp -f in Hadoop's dfs in the cp and copyFromLocal commands.  I've added HADOOP-9381 with a patch to document this feature in both the help page and the web page.

While I was looking at this, I realized that the mv and moveFromLocal commands didn't recognize the -f option, even though Unix's mv command does. Since it was simple to add, I created HADOOP-9382 with a patch to address that issue.