How to move S3 data to one partition level up?









up vote
0
down vote

favorite












Let's say I have a table in the path



s3:bucketname/tablename/month/day/deviceid


I want to move all files to s3:bucketname/tablename/month/day/. In other words, I want to ignore the last partition.



Note that there are already data in the path s3:bucketname/tablename/month/day/



Can I use aws s3 to achieve this?



I have more than 100K files so I cannot do this manually.










share|improve this question























  • Are they the actual names of the directories, or would they really be something like s3://bucketname/tablename/month=01/day=23/?
    – John Rotenstein
    Nov 12 at 18:45














up vote
0
down vote

favorite












Let's say I have a table in the path



s3:bucketname/tablename/month/day/deviceid


I want to move all files to s3:bucketname/tablename/month/day/. In other words, I want to ignore the last partition.



Note that there are already data in the path s3:bucketname/tablename/month/day/



Can I use aws s3 to achieve this?



I have more than 100K files so I cannot do this manually.










share|improve this question























  • Are they the actual names of the directories, or would they really be something like s3://bucketname/tablename/month=01/day=23/?
    – John Rotenstein
    Nov 12 at 18:45












up vote
0
down vote

favorite









up vote
0
down vote

favorite











Let's say I have a table in the path



s3:bucketname/tablename/month/day/deviceid


I want to move all files to s3:bucketname/tablename/month/day/. In other words, I want to ignore the last partition.



Note that there are already data in the path s3:bucketname/tablename/month/day/



Can I use aws s3 to achieve this?



I have more than 100K files so I cannot do this manually.










share|improve this question















Let's say I have a table in the path



s3:bucketname/tablename/month/day/deviceid


I want to move all files to s3:bucketname/tablename/month/day/. In other words, I want to ignore the last partition.



Note that there are already data in the path s3:bucketname/tablename/month/day/



Can I use aws s3 to achieve this?



I have more than 100K files so I cannot do this manually.







amazon-web-services amazon-s3






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 12 at 17:52

























asked Nov 12 at 6:44









John Hass

6171921




6171921











  • Are they the actual names of the directories, or would they really be something like s3://bucketname/tablename/month=01/day=23/?
    – John Rotenstein
    Nov 12 at 18:45
















  • Are they the actual names of the directories, or would they really be something like s3://bucketname/tablename/month=01/day=23/?
    – John Rotenstein
    Nov 12 at 18:45















Are they the actual names of the directories, or would they really be something like s3://bucketname/tablename/month=01/day=23/?
– John Rotenstein
Nov 12 at 18:45




Are they the actual names of the directories, or would they really be something like s3://bucketname/tablename/month=01/day=23/?
– John Rotenstein
Nov 12 at 18:45












2 Answers
2






active

oldest

votes

















up vote
0
down vote













I believe you can use the AWS console on the web. Open the source folder (deviceid), select all its contents by the checkbox to the left on Name.



You can select all files with a single click



Then select Actions → Copy.





Open the destination folder (day), click Actions → Paste. Confirm the operation.





Once the operation has finished successfully, you can delete the source folder with all its contents.






share|improve this answer



























    up vote
    0
    down vote













    You can do this in 2 ways.



    If you are not familiar with the aws cli, then go to the console, select all the files, cut and paste on the desired directory. However, this is very time consuming and inefficient task.



    I recommend the aws cli for S3. You can find some sample commands here.






    share|improve this answer




















      Your Answer






      StackExchange.ifUsing("editor", function ()
      StackExchange.using("externalEditor", function ()
      StackExchange.using("snippets", function ()
      StackExchange.snippets.init();
      );
      );
      , "code-snippets");

      StackExchange.ready(function()
      var channelOptions =
      tags: "".split(" "),
      id: "1"
      ;
      initTagRenderer("".split(" "), "".split(" "), channelOptions);

      StackExchange.using("externalEditor", function()
      // Have to fire editor after snippets, if snippets enabled
      if (StackExchange.settings.snippets.snippetsEnabled)
      StackExchange.using("snippets", function()
      createEditor();
      );

      else
      createEditor();

      );

      function createEditor()
      StackExchange.prepareEditor(
      heartbeatType: 'answer',
      autoActivateHeartbeat: false,
      convertImagesToLinks: true,
      noModals: true,
      showLowRepImageUploadWarning: true,
      reputationToPostImages: 10,
      bindNavPrevention: true,
      postfix: "",
      imageUploader:
      brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
      contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
      allowUrls: true
      ,
      onDemand: true,
      discardSelector: ".discard-answer"
      ,immediatelyShowMarkdownHelp:true
      );



      );













      draft saved

      draft discarded


















      StackExchange.ready(
      function ()
      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53257079%2fhow-to-move-s3-data-to-one-partition-level-up%23new-answer', 'question_page');

      );

      Post as a guest















      Required, but never shown

























      2 Answers
      2






      active

      oldest

      votes








      2 Answers
      2






      active

      oldest

      votes









      active

      oldest

      votes






      active

      oldest

      votes








      up vote
      0
      down vote













      I believe you can use the AWS console on the web. Open the source folder (deviceid), select all its contents by the checkbox to the left on Name.



      You can select all files with a single click



      Then select Actions → Copy.





      Open the destination folder (day), click Actions → Paste. Confirm the operation.





      Once the operation has finished successfully, you can delete the source folder with all its contents.






      share|improve this answer
























        up vote
        0
        down vote













        I believe you can use the AWS console on the web. Open the source folder (deviceid), select all its contents by the checkbox to the left on Name.



        You can select all files with a single click



        Then select Actions → Copy.





        Open the destination folder (day), click Actions → Paste. Confirm the operation.





        Once the operation has finished successfully, you can delete the source folder with all its contents.






        share|improve this answer






















          up vote
          0
          down vote










          up vote
          0
          down vote









          I believe you can use the AWS console on the web. Open the source folder (deviceid), select all its contents by the checkbox to the left on Name.



          You can select all files with a single click



          Then select Actions → Copy.





          Open the destination folder (day), click Actions → Paste. Confirm the operation.





          Once the operation has finished successfully, you can delete the source folder with all its contents.






          share|improve this answer












          I believe you can use the AWS console on the web. Open the source folder (deviceid), select all its contents by the checkbox to the left on Name.



          You can select all files with a single click



          Then select Actions → Copy.





          Open the destination folder (day), click Actions → Paste. Confirm the operation.





          Once the operation has finished successfully, you can delete the source folder with all its contents.







          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered Nov 12 at 6:52









          Yakov M.

          185




          185






















              up vote
              0
              down vote













              You can do this in 2 ways.



              If you are not familiar with the aws cli, then go to the console, select all the files, cut and paste on the desired directory. However, this is very time consuming and inefficient task.



              I recommend the aws cli for S3. You can find some sample commands here.






              share|improve this answer
























                up vote
                0
                down vote













                You can do this in 2 ways.



                If you are not familiar with the aws cli, then go to the console, select all the files, cut and paste on the desired directory. However, this is very time consuming and inefficient task.



                I recommend the aws cli for S3. You can find some sample commands here.






                share|improve this answer






















                  up vote
                  0
                  down vote










                  up vote
                  0
                  down vote









                  You can do this in 2 ways.



                  If you are not familiar with the aws cli, then go to the console, select all the files, cut and paste on the desired directory. However, this is very time consuming and inefficient task.



                  I recommend the aws cli for S3. You can find some sample commands here.






                  share|improve this answer












                  You can do this in 2 ways.



                  If you are not familiar with the aws cli, then go to the console, select all the files, cut and paste on the desired directory. However, this is very time consuming and inefficient task.



                  I recommend the aws cli for S3. You can find some sample commands here.







                  share|improve this answer












                  share|improve this answer



                  share|improve this answer










                  answered Nov 12 at 7:34









                  SAUJ

                  4171313




                  4171313



























                      draft saved

                      draft discarded
















































                      Thanks for contributing an answer to Stack Overflow!


                      • Please be sure to answer the question. Provide details and share your research!

                      But avoid


                      • Asking for help, clarification, or responding to other answers.

                      • Making statements based on opinion; back them up with references or personal experience.

                      To learn more, see our tips on writing great answers.





                      Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


                      Please pay close attention to the following guidance:


                      • Please be sure to answer the question. Provide details and share your research!

                      But avoid


                      • Asking for help, clarification, or responding to other answers.

                      • Making statements based on opinion; back them up with references or personal experience.

                      To learn more, see our tips on writing great answers.




                      draft saved


                      draft discarded














                      StackExchange.ready(
                      function ()
                      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53257079%2fhow-to-move-s3-data-to-one-partition-level-up%23new-answer', 'question_page');

                      );

                      Post as a guest















                      Required, but never shown





















































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown

































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown







                      Popular posts from this blog

                      Top Tejano songwriter Luis Silva dead of heart attack at 64

                      政党

                      天津地下鉄3号線