Transform an Uploaded CSV File Upload with Flask










2















I have a simple Flask app hosted on Python Anywhere. The app provides an interface where a user can upload a CSV file, and the app inserts a new column into the uploaded CSV file by overwriting the first column (row[0]) and putting the original row[0] at the end of the file and downloads the transformed file for the user. So, the CSV input looks like this (first row is a header row that should be unmodified save for the addition of a 4th column name):



Pet1,Pet2,Pet3
Cats,Dogs,Mice


And the output should look like:



Pet1,Pet2,Pet3,Pet4
Birds,Dogs,Mice,Cats


Here is my code:



from flask import Flask, make_response, request
import operator
import io
import csv

app = Flask(__name__)

def transform(text_file_contents):
reader = csv.reader(text_file_contents)
all =
row = next(reader)
row.append('Pet4')
all.append(row)
for row in reader:
pet4 = row[0]
row[0] = "Birds"
row.append(pet4)
all.append(row)
sortedlist = sorted(all, key=operator.itemgetter(0))
return sortedlist


@app.route('/')
def form():
return """
<html>
<body>
<h1>Upload a CSV File</h1>

<form action="/transform" method="post" enctype="multipart/form-data">
<input type="file" name="data_file" />
<input type="submit" />
</form>
</body>
</html>
"""

@app.route('/transform', methods=["POST"])
def transform_view():
request_file = request.files['data_file']
if not request_file:
return "No file"

file_contents = request_file.stream.read().decode("utf-8")

result = transform(file_contents)

response = make_response(result)
response.headers["Content-Disposition"] = "attachment; filename=result.csv"
return response


This is the error I am getting:



Traceback (most recent call last):
File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
raise value
File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/home/user/mysite/callnumbers.py", line 62, in transform_view
result = transform(file_contents)
File "/home/user/mysite/callnumbers.py", line 18, in transform
row[0] = row[1]
IndexError: list index out of range


I can tell that my script isn't reading the uploaded input file correctly (hence not being able to find the data in the columns I am trying to reference and the IndexError) but I haven't been able to find any examples of Flask using the Python CSV library to parse an uploaded file and manipulate columns in the way I need to. How can I resolve this? Any pointers in the right direction are much appreciated!










share|improve this question



















  • 1





    According to this question, the solution is to do request_file.read() instead of request_file.stream.read(). I've never done it that way myself but it seemed to work for that question's OP.

    – noslenkwah
    Nov 14 '18 at 22:44















2















I have a simple Flask app hosted on Python Anywhere. The app provides an interface where a user can upload a CSV file, and the app inserts a new column into the uploaded CSV file by overwriting the first column (row[0]) and putting the original row[0] at the end of the file and downloads the transformed file for the user. So, the CSV input looks like this (first row is a header row that should be unmodified save for the addition of a 4th column name):



Pet1,Pet2,Pet3
Cats,Dogs,Mice


And the output should look like:



Pet1,Pet2,Pet3,Pet4
Birds,Dogs,Mice,Cats


Here is my code:



from flask import Flask, make_response, request
import operator
import io
import csv

app = Flask(__name__)

def transform(text_file_contents):
reader = csv.reader(text_file_contents)
all =
row = next(reader)
row.append('Pet4')
all.append(row)
for row in reader:
pet4 = row[0]
row[0] = "Birds"
row.append(pet4)
all.append(row)
sortedlist = sorted(all, key=operator.itemgetter(0))
return sortedlist


@app.route('/')
def form():
return """
<html>
<body>
<h1>Upload a CSV File</h1>

<form action="/transform" method="post" enctype="multipart/form-data">
<input type="file" name="data_file" />
<input type="submit" />
</form>
</body>
</html>
"""

@app.route('/transform', methods=["POST"])
def transform_view():
request_file = request.files['data_file']
if not request_file:
return "No file"

file_contents = request_file.stream.read().decode("utf-8")

result = transform(file_contents)

response = make_response(result)
response.headers["Content-Disposition"] = "attachment; filename=result.csv"
return response


This is the error I am getting:



Traceback (most recent call last):
File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
raise value
File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/home/user/mysite/callnumbers.py", line 62, in transform_view
result = transform(file_contents)
File "/home/user/mysite/callnumbers.py", line 18, in transform
row[0] = row[1]
IndexError: list index out of range


I can tell that my script isn't reading the uploaded input file correctly (hence not being able to find the data in the columns I am trying to reference and the IndexError) but I haven't been able to find any examples of Flask using the Python CSV library to parse an uploaded file and manipulate columns in the way I need to. How can I resolve this? Any pointers in the right direction are much appreciated!










share|improve this question



















  • 1





    According to this question, the solution is to do request_file.read() instead of request_file.stream.read(). I've never done it that way myself but it seemed to work for that question's OP.

    – noslenkwah
    Nov 14 '18 at 22:44













2












2








2








I have a simple Flask app hosted on Python Anywhere. The app provides an interface where a user can upload a CSV file, and the app inserts a new column into the uploaded CSV file by overwriting the first column (row[0]) and putting the original row[0] at the end of the file and downloads the transformed file for the user. So, the CSV input looks like this (first row is a header row that should be unmodified save for the addition of a 4th column name):



Pet1,Pet2,Pet3
Cats,Dogs,Mice


And the output should look like:



Pet1,Pet2,Pet3,Pet4
Birds,Dogs,Mice,Cats


Here is my code:



from flask import Flask, make_response, request
import operator
import io
import csv

app = Flask(__name__)

def transform(text_file_contents):
reader = csv.reader(text_file_contents)
all =
row = next(reader)
row.append('Pet4')
all.append(row)
for row in reader:
pet4 = row[0]
row[0] = "Birds"
row.append(pet4)
all.append(row)
sortedlist = sorted(all, key=operator.itemgetter(0))
return sortedlist


@app.route('/')
def form():
return """
<html>
<body>
<h1>Upload a CSV File</h1>

<form action="/transform" method="post" enctype="multipart/form-data">
<input type="file" name="data_file" />
<input type="submit" />
</form>
</body>
</html>
"""

@app.route('/transform', methods=["POST"])
def transform_view():
request_file = request.files['data_file']
if not request_file:
return "No file"

file_contents = request_file.stream.read().decode("utf-8")

result = transform(file_contents)

response = make_response(result)
response.headers["Content-Disposition"] = "attachment; filename=result.csv"
return response


This is the error I am getting:



Traceback (most recent call last):
File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
raise value
File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/home/user/mysite/callnumbers.py", line 62, in transform_view
result = transform(file_contents)
File "/home/user/mysite/callnumbers.py", line 18, in transform
row[0] = row[1]
IndexError: list index out of range


I can tell that my script isn't reading the uploaded input file correctly (hence not being able to find the data in the columns I am trying to reference and the IndexError) but I haven't been able to find any examples of Flask using the Python CSV library to parse an uploaded file and manipulate columns in the way I need to. How can I resolve this? Any pointers in the right direction are much appreciated!










share|improve this question
















I have a simple Flask app hosted on Python Anywhere. The app provides an interface where a user can upload a CSV file, and the app inserts a new column into the uploaded CSV file by overwriting the first column (row[0]) and putting the original row[0] at the end of the file and downloads the transformed file for the user. So, the CSV input looks like this (first row is a header row that should be unmodified save for the addition of a 4th column name):



Pet1,Pet2,Pet3
Cats,Dogs,Mice


And the output should look like:



Pet1,Pet2,Pet3,Pet4
Birds,Dogs,Mice,Cats


Here is my code:



from flask import Flask, make_response, request
import operator
import io
import csv

app = Flask(__name__)

def transform(text_file_contents):
reader = csv.reader(text_file_contents)
all =
row = next(reader)
row.append('Pet4')
all.append(row)
for row in reader:
pet4 = row[0]
row[0] = "Birds"
row.append(pet4)
all.append(row)
sortedlist = sorted(all, key=operator.itemgetter(0))
return sortedlist


@app.route('/')
def form():
return """
<html>
<body>
<h1>Upload a CSV File</h1>

<form action="/transform" method="post" enctype="multipart/form-data">
<input type="file" name="data_file" />
<input type="submit" />
</form>
</body>
</html>
"""

@app.route('/transform', methods=["POST"])
def transform_view():
request_file = request.files['data_file']
if not request_file:
return "No file"

file_contents = request_file.stream.read().decode("utf-8")

result = transform(file_contents)

response = make_response(result)
response.headers["Content-Disposition"] = "attachment; filename=result.csv"
return response


This is the error I am getting:



Traceback (most recent call last):
File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
raise value
File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "/home/user/.virtualenvs/myvirtualenv/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/home/user/mysite/callnumbers.py", line 62, in transform_view
result = transform(file_contents)
File "/home/user/mysite/callnumbers.py", line 18, in transform
row[0] = row[1]
IndexError: list index out of range


I can tell that my script isn't reading the uploaded input file correctly (hence not being able to find the data in the columns I am trying to reference and the IndexError) but I haven't been able to find any examples of Flask using the Python CSV library to parse an uploaded file and manipulate columns in the way I need to. How can I resolve this? Any pointers in the right direction are much appreciated!







python csv flask pythonanywhere






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 14 '18 at 23:01









Joel

1,5706719




1,5706719










asked Nov 14 '18 at 22:30









lpmlpm

156




156







  • 1





    According to this question, the solution is to do request_file.read() instead of request_file.stream.read(). I've never done it that way myself but it seemed to work for that question's OP.

    – noslenkwah
    Nov 14 '18 at 22:44












  • 1





    According to this question, the solution is to do request_file.read() instead of request_file.stream.read(). I've never done it that way myself but it seemed to work for that question's OP.

    – noslenkwah
    Nov 14 '18 at 22:44







1




1





According to this question, the solution is to do request_file.read() instead of request_file.stream.read(). I've never done it that way myself but it seemed to work for that question's OP.

– noslenkwah
Nov 14 '18 at 22:44





According to this question, the solution is to do request_file.read() instead of request_file.stream.read(). I've never done it that way myself but it seemed to work for that question's OP.

– noslenkwah
Nov 14 '18 at 22:44












1 Answer
1






active

oldest

votes


















0














The csv.reader() function takes a file object instead of text. One way to do it is to save the file then reopen it in the transform function.



from flask import Flask, make_response, request
import operator
import io
import csv

app = Flask(__name__)


def transform():
with open('myfile.csv', 'rb') as f:
reader = csv.reader(f)
all =
row = next(reader)
row.append('Pet4')
all.append(row)
for row in reader:
pet4 = row[0]
row[0] = "Birds"
row.append(pet4)
all.append(row)
sortedlist = sorted(all, key=operator.itemgetter(0))
return sortedlist

@app.route('/')
def form():
return """
<html>
<body>
<h1>Upload a CSV File</h1>

<form action="/transform" method="post" enctype="multipart/form-data">
<input type="file" name="data_file" />
<input type="submit" />
</form>
</body>
</html>
"""

@app.route('/transform', methods=["POST"])
def transform_view():
request_file = request.files['data_file']
request_file.save("myfile.csv")
if not request_file:
return "No file"
#file_contents = io.StringIO(request_file.stream.read().decode("UTF8"), newline=None)
#file_contents = request_file.stream.read().decode("utf-8")

result = transform()
print result

response = make_response(result)
response.headers["Content-Disposition"] = "attachment; filename=result.csv"
return response
if __name__ == '__main__':
app.run()





share|improve this answer
























    Your Answer






    StackExchange.ifUsing("editor", function ()
    StackExchange.using("externalEditor", function ()
    StackExchange.using("snippets", function ()
    StackExchange.snippets.init();
    );
    );
    , "code-snippets");

    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "1"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader:
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    ,
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );













    draft saved

    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53309707%2ftransform-an-uploaded-csv-file-upload-with-flask%23new-answer', 'question_page');

    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    0














    The csv.reader() function takes a file object instead of text. One way to do it is to save the file then reopen it in the transform function.



    from flask import Flask, make_response, request
    import operator
    import io
    import csv

    app = Flask(__name__)


    def transform():
    with open('myfile.csv', 'rb') as f:
    reader = csv.reader(f)
    all =
    row = next(reader)
    row.append('Pet4')
    all.append(row)
    for row in reader:
    pet4 = row[0]
    row[0] = "Birds"
    row.append(pet4)
    all.append(row)
    sortedlist = sorted(all, key=operator.itemgetter(0))
    return sortedlist

    @app.route('/')
    def form():
    return """
    <html>
    <body>
    <h1>Upload a CSV File</h1>

    <form action="/transform" method="post" enctype="multipart/form-data">
    <input type="file" name="data_file" />
    <input type="submit" />
    </form>
    </body>
    </html>
    """

    @app.route('/transform', methods=["POST"])
    def transform_view():
    request_file = request.files['data_file']
    request_file.save("myfile.csv")
    if not request_file:
    return "No file"
    #file_contents = io.StringIO(request_file.stream.read().decode("UTF8"), newline=None)
    #file_contents = request_file.stream.read().decode("utf-8")

    result = transform()
    print result

    response = make_response(result)
    response.headers["Content-Disposition"] = "attachment; filename=result.csv"
    return response
    if __name__ == '__main__':
    app.run()





    share|improve this answer





























      0














      The csv.reader() function takes a file object instead of text. One way to do it is to save the file then reopen it in the transform function.



      from flask import Flask, make_response, request
      import operator
      import io
      import csv

      app = Flask(__name__)


      def transform():
      with open('myfile.csv', 'rb') as f:
      reader = csv.reader(f)
      all =
      row = next(reader)
      row.append('Pet4')
      all.append(row)
      for row in reader:
      pet4 = row[0]
      row[0] = "Birds"
      row.append(pet4)
      all.append(row)
      sortedlist = sorted(all, key=operator.itemgetter(0))
      return sortedlist

      @app.route('/')
      def form():
      return """
      <html>
      <body>
      <h1>Upload a CSV File</h1>

      <form action="/transform" method="post" enctype="multipart/form-data">
      <input type="file" name="data_file" />
      <input type="submit" />
      </form>
      </body>
      </html>
      """

      @app.route('/transform', methods=["POST"])
      def transform_view():
      request_file = request.files['data_file']
      request_file.save("myfile.csv")
      if not request_file:
      return "No file"
      #file_contents = io.StringIO(request_file.stream.read().decode("UTF8"), newline=None)
      #file_contents = request_file.stream.read().decode("utf-8")

      result = transform()
      print result

      response = make_response(result)
      response.headers["Content-Disposition"] = "attachment; filename=result.csv"
      return response
      if __name__ == '__main__':
      app.run()





      share|improve this answer



























        0












        0








        0







        The csv.reader() function takes a file object instead of text. One way to do it is to save the file then reopen it in the transform function.



        from flask import Flask, make_response, request
        import operator
        import io
        import csv

        app = Flask(__name__)


        def transform():
        with open('myfile.csv', 'rb') as f:
        reader = csv.reader(f)
        all =
        row = next(reader)
        row.append('Pet4')
        all.append(row)
        for row in reader:
        pet4 = row[0]
        row[0] = "Birds"
        row.append(pet4)
        all.append(row)
        sortedlist = sorted(all, key=operator.itemgetter(0))
        return sortedlist

        @app.route('/')
        def form():
        return """
        <html>
        <body>
        <h1>Upload a CSV File</h1>

        <form action="/transform" method="post" enctype="multipart/form-data">
        <input type="file" name="data_file" />
        <input type="submit" />
        </form>
        </body>
        </html>
        """

        @app.route('/transform', methods=["POST"])
        def transform_view():
        request_file = request.files['data_file']
        request_file.save("myfile.csv")
        if not request_file:
        return "No file"
        #file_contents = io.StringIO(request_file.stream.read().decode("UTF8"), newline=None)
        #file_contents = request_file.stream.read().decode("utf-8")

        result = transform()
        print result

        response = make_response(result)
        response.headers["Content-Disposition"] = "attachment; filename=result.csv"
        return response
        if __name__ == '__main__':
        app.run()





        share|improve this answer















        The csv.reader() function takes a file object instead of text. One way to do it is to save the file then reopen it in the transform function.



        from flask import Flask, make_response, request
        import operator
        import io
        import csv

        app = Flask(__name__)


        def transform():
        with open('myfile.csv', 'rb') as f:
        reader = csv.reader(f)
        all =
        row = next(reader)
        row.append('Pet4')
        all.append(row)
        for row in reader:
        pet4 = row[0]
        row[0] = "Birds"
        row.append(pet4)
        all.append(row)
        sortedlist = sorted(all, key=operator.itemgetter(0))
        return sortedlist

        @app.route('/')
        def form():
        return """
        <html>
        <body>
        <h1>Upload a CSV File</h1>

        <form action="/transform" method="post" enctype="multipart/form-data">
        <input type="file" name="data_file" />
        <input type="submit" />
        </form>
        </body>
        </html>
        """

        @app.route('/transform', methods=["POST"])
        def transform_view():
        request_file = request.files['data_file']
        request_file.save("myfile.csv")
        if not request_file:
        return "No file"
        #file_contents = io.StringIO(request_file.stream.read().decode("UTF8"), newline=None)
        #file_contents = request_file.stream.read().decode("utf-8")

        result = transform()
        print result

        response = make_response(result)
        response.headers["Content-Disposition"] = "attachment; filename=result.csv"
        return response
        if __name__ == '__main__':
        app.run()






        share|improve this answer














        share|improve this answer



        share|improve this answer








        edited Nov 15 '18 at 17:03

























        answered Nov 15 '18 at 15:19









        Mike C.Mike C.

        417316




        417316





























            draft saved

            draft discarded
















































            Thanks for contributing an answer to Stack Overflow!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid


            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.

            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53309707%2ftransform-an-uploaded-csv-file-upload-with-flask%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Top Tejano songwriter Luis Silva dead of heart attack at 64

            政党

            天津地下鉄3号線