How to copy files from Windows to Linux HDFS directly












1















I am trying to copy a large amount files to Hadoop HIVE. Right now I am doing this via a two-step scripts.




  • Step 1: Powershell scripts copying files from Windows to Linux using
    Putty scp tool.


  • Step 2: Bash scripts copying files from Linux to HDFS using Hadoop
    put command.



Is there a way to do this via just one step?










share|improve this question























  • I tried out HortonWorks' Sandbox. Via Ambari I can upload files (one file a time) directly into HDFS. I wonder whether HortonWorks provide any tool to automate this process.

    – Seaport
    Mar 1 at 0:04
















1















I am trying to copy a large amount files to Hadoop HIVE. Right now I am doing this via a two-step scripts.




  • Step 1: Powershell scripts copying files from Windows to Linux using
    Putty scp tool.


  • Step 2: Bash scripts copying files from Linux to HDFS using Hadoop
    put command.



Is there a way to do this via just one step?










share|improve this question























  • I tried out HortonWorks' Sandbox. Via Ambari I can upload files (one file a time) directly into HDFS. I wonder whether HortonWorks provide any tool to automate this process.

    – Seaport
    Mar 1 at 0:04














1












1








1








I am trying to copy a large amount files to Hadoop HIVE. Right now I am doing this via a two-step scripts.




  • Step 1: Powershell scripts copying files from Windows to Linux using
    Putty scp tool.


  • Step 2: Bash scripts copying files from Linux to HDFS using Hadoop
    put command.



Is there a way to do this via just one step?










share|improve this question














I am trying to copy a large amount files to Hadoop HIVE. Right now I am doing this via a two-step scripts.




  • Step 1: Powershell scripts copying files from Windows to Linux using
    Putty scp tool.


  • Step 2: Bash scripts copying files from Linux to HDFS using Hadoop
    put command.



Is there a way to do this via just one step?







linux windows scp hadoop dfs






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Feb 8 at 21:19









SeaportSeaport

72




72













  • I tried out HortonWorks' Sandbox. Via Ambari I can upload files (one file a time) directly into HDFS. I wonder whether HortonWorks provide any tool to automate this process.

    – Seaport
    Mar 1 at 0:04



















  • I tried out HortonWorks' Sandbox. Via Ambari I can upload files (one file a time) directly into HDFS. I wonder whether HortonWorks provide any tool to automate this process.

    – Seaport
    Mar 1 at 0:04

















I tried out HortonWorks' Sandbox. Via Ambari I can upload files (one file a time) directly into HDFS. I wonder whether HortonWorks provide any tool to automate this process.

– Seaport
Mar 1 at 0:04





I tried out HortonWorks' Sandbox. Via Ambari I can upload files (one file a time) directly into HDFS. I wonder whether HortonWorks provide any tool to automate this process.

– Seaport
Mar 1 at 0:04










0






active

oldest

votes











Your Answer








StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "3"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1403680%2fhow-to-copy-files-from-windows-to-linux-hdfs-directly%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























0






active

oldest

votes








0






active

oldest

votes









active

oldest

votes






active

oldest

votes
















draft saved

draft discarded




















































Thanks for contributing an answer to Super User!


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1403680%2fhow-to-copy-files-from-windows-to-linux-hdfs-directly%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Probability when a professor distributes a quiz and homework assignment to a class of n students.

Aardman Animations

Are they similar matrix