how to handle await calls in async loop functions - nodejs
I am reading a CSV using CSV-parser npm module and have to perform some operation on the data I get from the CSV (for each line).
const readstream = fs.createReadStream('src/working_file.csv');
const stream = readstream.pipe(parser());
stream.on('data', async data =>
// data is a JSON object of the row in CSV.
// Now i am calling another async function from user using the data in the JSON
console.log('before calling');
const writeToFile = await getImage(data.searchKey);
console.log('after calling');
// do other stuff
async function getImage(searchKey)
// im doing web scraping here using puppeeter
// it has some await calls too
console.log('in getimage');
const results = await scrapper.run().catch(err =>
console.error(err);
process.exit(1);
);
let say my csv has 2 rows then, my output is coming like below
before calling
in getimage
before calling
in getimage
after calling
after calling
but when I am doing this all callings are happening at a time though I used await. If I have 10 rows in the CSV all 10 rows calling the function is happening at the same time. but I want it to happen one by one. Only when the operation with the first row completes then I want the operate the second row.
my problem is all calls are happening at once rather than once by one.
javascript node.js asynchronous fs
|
show 4 more comments
I am reading a CSV using CSV-parser npm module and have to perform some operation on the data I get from the CSV (for each line).
const readstream = fs.createReadStream('src/working_file.csv');
const stream = readstream.pipe(parser());
stream.on('data', async data =>
// data is a JSON object of the row in CSV.
// Now i am calling another async function from user using the data in the JSON
console.log('before calling');
const writeToFile = await getImage(data.searchKey);
console.log('after calling');
// do other stuff
async function getImage(searchKey)
// im doing web scraping here using puppeeter
// it has some await calls too
console.log('in getimage');
const results = await scrapper.run().catch(err =>
console.error(err);
process.exit(1);
);
let say my csv has 2 rows then, my output is coming like below
before calling
in getimage
before calling
in getimage
after calling
after calling
but when I am doing this all callings are happening at a time though I used await. If I have 10 rows in the CSV all 10 rows calling the function is happening at the same time. but I want it to happen one by one. Only when the operation with the first row completes then I want the operate the second row.
my problem is all calls are happening at once rather than once by one.
javascript node.js asynchronous fs
Can you show the code dealing with ten rows?
– Chris G
Nov 14 '18 at 10:41
Node.js works in the same way as other js platforms. See stackoverflow.com/questions/14220321/…
– David Lemon
Nov 14 '18 at 10:44
Possible duplicate of How do I return the response from an asynchronous call?
– David Lemon
Nov 14 '18 at 10:44
David, I have explained my problem more clear
– m9m9m
Nov 14 '18 at 11:28
you would need to provide thegetImage
code maybe there is something in there, maybe someawait
is needed
– Nikos M.
Nov 14 '18 at 12:00
|
show 4 more comments
I am reading a CSV using CSV-parser npm module and have to perform some operation on the data I get from the CSV (for each line).
const readstream = fs.createReadStream('src/working_file.csv');
const stream = readstream.pipe(parser());
stream.on('data', async data =>
// data is a JSON object of the row in CSV.
// Now i am calling another async function from user using the data in the JSON
console.log('before calling');
const writeToFile = await getImage(data.searchKey);
console.log('after calling');
// do other stuff
async function getImage(searchKey)
// im doing web scraping here using puppeeter
// it has some await calls too
console.log('in getimage');
const results = await scrapper.run().catch(err =>
console.error(err);
process.exit(1);
);
let say my csv has 2 rows then, my output is coming like below
before calling
in getimage
before calling
in getimage
after calling
after calling
but when I am doing this all callings are happening at a time though I used await. If I have 10 rows in the CSV all 10 rows calling the function is happening at the same time. but I want it to happen one by one. Only when the operation with the first row completes then I want the operate the second row.
my problem is all calls are happening at once rather than once by one.
javascript node.js asynchronous fs
I am reading a CSV using CSV-parser npm module and have to perform some operation on the data I get from the CSV (for each line).
const readstream = fs.createReadStream('src/working_file.csv');
const stream = readstream.pipe(parser());
stream.on('data', async data =>
// data is a JSON object of the row in CSV.
// Now i am calling another async function from user using the data in the JSON
console.log('before calling');
const writeToFile = await getImage(data.searchKey);
console.log('after calling');
// do other stuff
async function getImage(searchKey)
// im doing web scraping here using puppeeter
// it has some await calls too
console.log('in getimage');
const results = await scrapper.run().catch(err =>
console.error(err);
process.exit(1);
);
let say my csv has 2 rows then, my output is coming like below
before calling
in getimage
before calling
in getimage
after calling
after calling
but when I am doing this all callings are happening at a time though I used await. If I have 10 rows in the CSV all 10 rows calling the function is happening at the same time. but I want it to happen one by one. Only when the operation with the first row completes then I want the operate the second row.
my problem is all calls are happening at once rather than once by one.
javascript node.js asynchronous fs
javascript node.js asynchronous fs
edited Nov 14 '18 at 12:28
m9m9m
asked Nov 14 '18 at 10:37
m9m9mm9m9m
497512
497512
Can you show the code dealing with ten rows?
– Chris G
Nov 14 '18 at 10:41
Node.js works in the same way as other js platforms. See stackoverflow.com/questions/14220321/…
– David Lemon
Nov 14 '18 at 10:44
Possible duplicate of How do I return the response from an asynchronous call?
– David Lemon
Nov 14 '18 at 10:44
David, I have explained my problem more clear
– m9m9m
Nov 14 '18 at 11:28
you would need to provide thegetImage
code maybe there is something in there, maybe someawait
is needed
– Nikos M.
Nov 14 '18 at 12:00
|
show 4 more comments
Can you show the code dealing with ten rows?
– Chris G
Nov 14 '18 at 10:41
Node.js works in the same way as other js platforms. See stackoverflow.com/questions/14220321/…
– David Lemon
Nov 14 '18 at 10:44
Possible duplicate of How do I return the response from an asynchronous call?
– David Lemon
Nov 14 '18 at 10:44
David, I have explained my problem more clear
– m9m9m
Nov 14 '18 at 11:28
you would need to provide thegetImage
code maybe there is something in there, maybe someawait
is needed
– Nikos M.
Nov 14 '18 at 12:00
Can you show the code dealing with ten rows?
– Chris G
Nov 14 '18 at 10:41
Can you show the code dealing with ten rows?
– Chris G
Nov 14 '18 at 10:41
Node.js works in the same way as other js platforms. See stackoverflow.com/questions/14220321/…
– David Lemon
Nov 14 '18 at 10:44
Node.js works in the same way as other js platforms. See stackoverflow.com/questions/14220321/…
– David Lemon
Nov 14 '18 at 10:44
Possible duplicate of How do I return the response from an asynchronous call?
– David Lemon
Nov 14 '18 at 10:44
Possible duplicate of How do I return the response from an asynchronous call?
– David Lemon
Nov 14 '18 at 10:44
David, I have explained my problem more clear
– m9m9m
Nov 14 '18 at 11:28
David, I have explained my problem more clear
– m9m9m
Nov 14 '18 at 11:28
you would need to provide the
getImage
code maybe there is something in there, maybe some await
is needed– Nikos M.
Nov 14 '18 at 12:00
you would need to provide the
getImage
code maybe there is something in there, maybe some await
is needed– Nikos M.
Nov 14 '18 at 12:00
|
show 4 more comments
1 Answer
1
active
oldest
votes
Try this code.
var fs = require('fs');
var parse = require('csv-parse');
var async = require('async');
var inputFile='src/working_file.csv';
var parser = parse(delimiter: ',', function (err, data)
async.eachSeries(data, function (line, callback)
// do something with the line
doSomething(line).then(function()
// when processing finishes invoke the callback to move to the next one
callback();
);
)
);
fs.createReadStream(inputFile).pipe(parser);
You can also use fast-csv
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53298211%2fhow-to-handle-await-calls-in-async-loop-functions-nodejs%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Try this code.
var fs = require('fs');
var parse = require('csv-parse');
var async = require('async');
var inputFile='src/working_file.csv';
var parser = parse(delimiter: ',', function (err, data)
async.eachSeries(data, function (line, callback)
// do something with the line
doSomething(line).then(function()
// when processing finishes invoke the callback to move to the next one
callback();
);
)
);
fs.createReadStream(inputFile).pipe(parser);
You can also use fast-csv
add a comment |
Try this code.
var fs = require('fs');
var parse = require('csv-parse');
var async = require('async');
var inputFile='src/working_file.csv';
var parser = parse(delimiter: ',', function (err, data)
async.eachSeries(data, function (line, callback)
// do something with the line
doSomething(line).then(function()
// when processing finishes invoke the callback to move to the next one
callback();
);
)
);
fs.createReadStream(inputFile).pipe(parser);
You can also use fast-csv
add a comment |
Try this code.
var fs = require('fs');
var parse = require('csv-parse');
var async = require('async');
var inputFile='src/working_file.csv';
var parser = parse(delimiter: ',', function (err, data)
async.eachSeries(data, function (line, callback)
// do something with the line
doSomething(line).then(function()
// when processing finishes invoke the callback to move to the next one
callback();
);
)
);
fs.createReadStream(inputFile).pipe(parser);
You can also use fast-csv
Try this code.
var fs = require('fs');
var parse = require('csv-parse');
var async = require('async');
var inputFile='src/working_file.csv';
var parser = parse(delimiter: ',', function (err, data)
async.eachSeries(data, function (line, callback)
// do something with the line
doSomething(line).then(function()
// when processing finishes invoke the callback to move to the next one
callback();
);
)
);
fs.createReadStream(inputFile).pipe(parser);
You can also use fast-csv
answered Nov 14 '18 at 10:51
Sayed Mohd AliSayed Mohd Ali
1,1172419
1,1172419
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53298211%2fhow-to-handle-await-calls-in-async-loop-functions-nodejs%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Can you show the code dealing with ten rows?
– Chris G
Nov 14 '18 at 10:41
Node.js works in the same way as other js platforms. See stackoverflow.com/questions/14220321/…
– David Lemon
Nov 14 '18 at 10:44
Possible duplicate of How do I return the response from an asynchronous call?
– David Lemon
Nov 14 '18 at 10:44
David, I have explained my problem more clear
– m9m9m
Nov 14 '18 at 11:28
you would need to provide the
getImage
code maybe there is something in there, maybe someawait
is needed– Nikos M.
Nov 14 '18 at 12:00