Async and LINQ Queries: Retrieve all lines in all files in a directory tree
up vote
0
down vote
favorite
I have some UWP code that I am trying to maintain. Its goal is to coalesce some plaintext files kept in a directory structure into a single XML file.
In the interest of doing this, I previously grab all distinct lines in all files in the dirrectory structure using LINQ:
Imports System.IO
Async Function getAllLines() As IEnumerable(Of String)
Dim _folderPicker As New Windows.Storage.Pickers.FolderPicker()
_folderPicker.FileTypeFilter.Add("*") 'We must add this; it is a known bug
Dim _rootDir As Windows.Storage.StorageFolder = Await _folderPicker.PickSingleFolderAsync()
Dim allDistinctLines As IEnumerable(Of String) = From _file In Directory.GetFiles(_rootDir.Name, "*.txt", SearchOption.AllDirectories) From line In File.ReadAllLines(_file) Where line.Trim().Length > 0 Select line.Trim() Distinct
Return allDistinctLines
End Function
Since late October, however, it seems some things have changed regarding file perrmissions (Source). I came back to the code after about a month of it working to find it throwing an exception saying that the folder that I was previously able to write to was not accessable due to permissions. The folder existed and I grabbed the folder from a picker so I thought that I was good. Previously, as long as I had grabbed a File or Folder using a picker, I could still use a System.IO function to enumerate, read from or write to said folder or directory.
Now, it seems as though you must go though the proper UWP faculties, which is fine, but I am unsure how to rewrite my LINQ query as the UWP methods are asynchronous.
Here is what I have tried:
Async Function getAllLines() As IEnumerable(Of String)
Dim _folderPicker As New Windows.Storage.Pickers.FolderPicker()
_folderPicker.FileTypeFilter.Add("*") 'We must add this; it is a known bug
Dim _rootDir As Windows.Storage.StorageFolder = Await _folderPicker.PickSingleFolderAsync()
Dim queryOptions As New Windows.Storage.Search.QueryOptions()
queryOptions.FileTypeFilter.Add(".txt")
queryOptions.FolderDepth = Windows.Storage.Search.FolderDepth.Deep
Dim allDistinctLines As IEnumerable(Of String) = From _file In Await _rootDir.CreateFileQueryWithOptions(queryOptions).GetFilesAsync() From line In Await Windows.Storage.FileIO.ReadLinesAsync(_file) Where line.Trim().Length > 0 Select line.Trim() Distinct
Return allDistinctLines
End Function
In the above code, I am at least able to enumerate the folder if I comment out the second await in the LINQ. The issue is that with the second Await in the LINQ query, I cannot compile as
'Await' may only be used in a query expression within the first collection expression of the initial 'From' clause or within the collection expression of a 'Join' clause.
So I read up on Join clauses, which seem to equate disparate data, which isn't really what I am looking for.
I read up on the error in general and it seems like the LINQ queries and Async Functions only have limited functionality. Honestly, I don't know enough about LINQ to really make that claim, but that was my feeling.
I found this answer, which uses Task.WhenAll() to facilitate the LINQ and Async Functions usage, but I can't wrap my head around exactly how it would be put into practice.
How to await a method in a Linq query
So I tried:
Async Function getAllLines() As IEnumerable(Of String)
Dim _folderPicker As New Windows.Storage.Pickers.FolderPicker()
_folderPicker.FileTypeFilter.Add("*") 'We must add this; it is a known bug
Dim _rootDir As Windows.Storage.StorageFolder = Await _folderPicker.PickSingleFolderAsync()
Dim queryOptions As New Windows.Storage.Search.QueryOptions()
queryOptions.FileTypeFilter.Add(".txt")
queryOptions.FolderDepth = Windows.Storage.Search.FolderDepth.Deep
Dim allFiles As IEnumerable(Of Windows.Storage.StorageFile) = From _file In Await _rootDir.CreateFileQueryWithOptions(queryOptions).GetFilesAsync() Select _file
Dim allDistinctLines As IEnumerable(Of String) = Task.WhenAll(allFiles.Select(Function(_file) Windows.Storage.FileIO.ReadLines.Async(_file).Where(Function(_line) _line.Trim().Length > 0).Distinct()))
Return allDistinctLines
End Function
But the return type isn't the same, it ends up being some strange IList(Of String)(), which doesn't work for me strangely. Maybe there is just a critical misunderstanding there.
Regardless, any help understanding Async file operations is appriciated!
vb.net linq uwp async-await
asked Nov 8 at 15:10
Tyler Gubala
184313
add a comment |
up vote
0
down vote
favorite
I have some UWP code that I am trying to maintain. Its goal is to coalesce some plaintext files kept in a directory structure into a single XML file.
In the interest of doing this, I previously grab all distinct lines in all files in the dirrectory structure using LINQ:
Imports System.IO
Async Function getAllLines() As IEnumerable(Of String)
Dim _folderPicker As New Windows.Storage.Pickers.FolderPicker()
_folderPicker.FileTypeFilter.Add("*") 'We must add this; it is a known bug
Dim _rootDir As Windows.Storage.StorageFolder = Await _folderPicker.PickSingleFolderAsync()
Dim allDistinctLines As IEnumerable(Of String) = From _file In Directory.GetFiles(_rootDir.Name, "*.txt", SearchOption.AllDirectories) From line In File.ReadAllLines(_file) Where line.Trim().Length > 0 Select line.Trim() Distinct
Return allDistinctLines
End Function
Since late October, however, it seems some things have changed regarding file perrmissions (Source). I came back to the code after about a month of it working to find it throwing an exception saying that the folder that I was previously able to write to was not accessable due to permissions. The folder existed and I grabbed the folder from a picker so I thought that I was good. Previously, as long as I had grabbed a File or Folder using a picker, I could still use a System.IO function to enumerate, read from or write to said folder or directory.
Now, it seems as though you must go though the proper UWP faculties, which is fine, but I am unsure how to rewrite my LINQ query as the UWP methods are asynchronous.
Here is what I have tried:
Async Function getAllLines() As IEnumerable(Of String)
Dim _folderPicker As New Windows.Storage.Pickers.FolderPicker()
_folderPicker.FileTypeFilter.Add("*") 'We must add this; it is a known bug
Dim _rootDir As Windows.Storage.StorageFolder = Await _folderPicker.PickSingleFolderAsync()
Dim queryOptions As New Windows.Storage.Search.QueryOptions()
queryOptions.FileTypeFilter.Add(".txt")
queryOptions.FolderDepth = Windows.Storage.Search.FolderDepth.Deep
Dim allDistinctLines As IEnumerable(Of String) = From _file In Await _rootDir.CreateFileQueryWithOptions(queryOptions).GetFilesAsync() From line In Await Windows.Storage.FileIO.ReadLinesAsync(_file) Where line.Trim().Length > 0 Select line.Trim() Distinct
Return allDistinctLines
End Function
In the above code, I am at least able to enumerate the folder if I comment out the second await in the LINQ. The issue is that with the second Await in the LINQ query, I cannot compile as
'Await' may only be used in a query expression within the first collection expression of the initial 'From' clause or within the collection expression of a 'Join' clause.
So I read up on Join clauses, which seem to equate disparate data, which isn't really what I am looking for.
I read up on the error in general and it seems like the LINQ queries and Async Functions only have limited functionality. Honestly, I don't know enough about LINQ to really make that claim, but that was my feeling.
I found this answer, which uses Task.WhenAll() to facilitate the LINQ and Async Functions usage, but I can't wrap my head around exactly how it would be put into practice.
How to await a method in a Linq query
So I tried:
Async Function getAllLines() As IEnumerable(Of String)
Dim _folderPicker As New Windows.Storage.Pickers.FolderPicker()
_folderPicker.FileTypeFilter.Add("*") 'We must add this; it is a known bug
Dim _rootDir As Windows.Storage.StorageFolder = Await _folderPicker.PickSingleFolderAsync()
Dim queryOptions As New Windows.Storage.Search.QueryOptions()
queryOptions.FileTypeFilter.Add(".txt")
queryOptions.FolderDepth = Windows.Storage.Search.FolderDepth.Deep
Dim allFiles As IEnumerable(Of Windows.Storage.StorageFile) = From _file In Await _rootDir.CreateFileQueryWithOptions(queryOptions).GetFilesAsync() Select _file
Dim allDistinctLines As IEnumerable(Of String) = Task.WhenAll(allFiles.Select(Function(_file) Windows.Storage.FileIO.ReadLines.Async(_file).Where(Function(_line) _line.Trim().Length > 0).Distinct()))
Return allDistinctLines
End Function
But the return type isn't the same, it ends up being some strange IList(Of String)(), which doesn't work for me strangely. Maybe there is just a critical misunderstanding there.
Regardless, any help understanding Async file operations is appriciated!
vb.net linq uwp async-await
asked Nov 8 at 15:10
Tyler Gubala
184313
Please check this case reply.
– CoCaIceDew
Nov 9 at 9:35
@CoCalceDew I have seen that post, and it was included in my original question. I need some clarifcation though on how it should be utilized. Any help would be appriciated.
– Tyler Gubala
Nov 12 at 17:21
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I have some UWP code that I am trying to maintain. Its goal is to coalesce some plaintext files kept in a directory structure into a single XML file.
In the interest of doing this, I previously grab all distinct lines in all files in the dirrectory structure using LINQ:
Imports System.IO
Async Function getAllLines() As IEnumerable(Of String)
Dim _folderPicker As New Windows.Storage.Pickers.FolderPicker()
_folderPicker.FileTypeFilter.Add("*") 'We must add this; it is a known bug
Dim _rootDir As Windows.Storage.StorageFolder = Await _folderPicker.PickSingleFolderAsync()
Dim allDistinctLines As IEnumerable(Of String) = From _file In Directory.GetFiles(_rootDir.Name, "*.txt", SearchOption.AllDirectories) From line In File.ReadAllLines(_file) Where line.Trim().Length > 0 Select line.Trim() Distinct
Return allDistinctLines
End Function
Since late October, however, it seems some things have changed regarding file perrmissions (Source). I came back to the code after about a month of it working to find it throwing an exception saying that the folder that I was previously able to write to was not accessable due to permissions. The folder existed and I grabbed the folder from a picker so I thought that I was good. Previously, as long as I had grabbed a File or Folder using a picker, I could still use a System.IO function to enumerate, read from or write to said folder or directory.
Now, it seems as though you must go though the proper UWP faculties, which is fine, but I am unsure how to rewrite my LINQ query as the UWP methods are asynchronous.
Here is what I have tried:
Async Function getAllLines() As IEnumerable(Of String)
Dim _folderPicker As New Windows.Storage.Pickers.FolderPicker()
_folderPicker.FileTypeFilter.Add("*") 'We must add this; it is a known bug
Dim _rootDir As Windows.Storage.StorageFolder = Await _folderPicker.PickSingleFolderAsync()
Dim queryOptions As New Windows.Storage.Search.QueryOptions()
queryOptions.FileTypeFilter.Add(".txt")
queryOptions.FolderDepth = Windows.Storage.Search.FolderDepth.Deep
Dim allDistinctLines As IEnumerable(Of String) = From _file In Await _rootDir.CreateFileQueryWithOptions(queryOptions).GetFilesAsync() From line In Await Windows.Storage.FileIO.ReadLinesAsync(_file) Where line.Trim().Length > 0 Select line.Trim() Distinct
Return allDistinctLines
End Function
In the above code, I am at least able to enumerate the folder if I comment out the second await in the LINQ. The issue is that with the second Await in the LINQ query, I cannot compile as
'Await' may only be used in a query expression within the first collection expression of the initial 'From' clause or within the collection expression of a 'Join' clause.
So I read up on Join clauses, which seem to equate disparate data, which isn't really what I am looking for.
I read up on the error in general and it seems like the LINQ queries and Async Functions only have limited functionality. Honestly, I don't know enough about LINQ to really make that claim, but that was my feeling.
I found this answer, which uses Task.WhenAll() to facilitate the LINQ and Async Functions usage, but I can't wrap my head around exactly how it would be put into practice.
How to await a method in a Linq query
So I tried:
Async Function getAllLines() As IEnumerable(Of String)
Dim _folderPicker As New Windows.Storage.Pickers.FolderPicker()
_folderPicker.FileTypeFilter.Add("*") 'We must add this; it is a known bug
Dim _rootDir As Windows.Storage.StorageFolder = Await _folderPicker.PickSingleFolderAsync()
Dim queryOptions As New Windows.Storage.Search.QueryOptions()
queryOptions.FileTypeFilter.Add(".txt")
queryOptions.FolderDepth = Windows.Storage.Search.FolderDepth.Deep
Dim allFiles As IEnumerable(Of Windows.Storage.StorageFile) = From _file In Await _rootDir.CreateFileQueryWithOptions(queryOptions).GetFilesAsync() Select _file
Dim allDistinctLines As IEnumerable(Of String) = Task.WhenAll(allFiles.Select(Function(_file) Windows.Storage.FileIO.ReadLines.Async(_file).Where(Function(_line) _line.Trim().Length > 0).Distinct()))
Return allDistinctLines
End Function
But the return type isn't the same, it ends up being some strange IList(Of String)(), which doesn't work for me strangely. Maybe there is just a critical misunderstanding there.
Regardless, any help understanding Async file operations is appriciated!
vb.net linq uwp async-await
asked Nov 8 at 15:10
Tyler Gubala
184313
I have some UWP code that I am trying to maintain. Its goal is to coalesce some plaintext files kept in a directory structure into a single XML file.
In the interest of doing this, I previously grab all distinct lines in all files in the dirrectory structure using LINQ:
Imports System.IO
Async Function getAllLines() As IEnumerable(Of String)
Dim _folderPicker As New Windows.Storage.Pickers.FolderPicker()
_folderPicker.FileTypeFilter.Add("*") 'We must add this; it is a known bug
Dim _rootDir As Windows.Storage.StorageFolder = Await _folderPicker.PickSingleFolderAsync()
Dim allDistinctLines As IEnumerable(Of String) = From _file In Directory.GetFiles(_rootDir.Name, "*.txt", SearchOption.AllDirectories) From line In File.ReadAllLines(_file) Where line.Trim().Length > 0 Select line.Trim() Distinct
Return allDistinctLines
End Function
Since late October, however, it seems some things have changed regarding file perrmissions (Source). I came back to the code after about a month of it working to find it throwing an exception saying that the folder that I was previously able to write to was not accessable due to permissions. The folder existed and I grabbed the folder from a picker so I thought that I was good. Previously, as long as I had grabbed a File or Folder using a picker, I could still use a System.IO function to enumerate, read from or write to said folder or directory.
Now, it seems as though you must go though the proper UWP faculties, which is fine, but I am unsure how to rewrite my LINQ query as the UWP methods are asynchronous.
Here is what I have tried:
Async Function getAllLines() As IEnumerable(Of String)
Dim _folderPicker As New Windows.Storage.Pickers.FolderPicker()
_folderPicker.FileTypeFilter.Add("*") 'We must add this; it is a known bug
Dim _rootDir As Windows.Storage.StorageFolder = Await _folderPicker.PickSingleFolderAsync()
Dim queryOptions As New Windows.Storage.Search.QueryOptions()
queryOptions.FileTypeFilter.Add(".txt")
queryOptions.FolderDepth = Windows.Storage.Search.FolderDepth.Deep
Dim allDistinctLines As IEnumerable(Of String) = From _file In Await _rootDir.CreateFileQueryWithOptions(queryOptions).GetFilesAsync() From line In Await Windows.Storage.FileIO.ReadLinesAsync(_file) Where line.Trim().Length > 0 Select line.Trim() Distinct
Return allDistinctLines
End Function
In the above code, I am at least able to enumerate the folder if I comment out the second await in the LINQ. The issue is that with the second Await in the LINQ query, I cannot compile as
'Await' may only be used in a query expression within the first collection expression of the initial 'From' clause or within the collection expression of a 'Join' clause.
So I read up on Join clauses, which seem to equate disparate data, which isn't really what I am looking for.
I read up on the error in general and it seems like the LINQ queries and Async Functions only have limited functionality. Honestly, I don't know enough about LINQ to really make that claim, but that was my feeling.
I found this answer, which uses Task.WhenAll() to facilitate the LINQ and Async Functions usage, but I can't wrap my head around exactly how it would be put into practice.
How to await a method in a Linq query
So I tried:
Async Function getAllLines() As IEnumerable(Of String)
Dim _folderPicker As New Windows.Storage.Pickers.FolderPicker()
_folderPicker.FileTypeFilter.Add("*") 'We must add this; it is a known bug
Dim _rootDir As Windows.Storage.StorageFolder = Await _folderPicker.PickSingleFolderAsync()
Dim queryOptions As New Windows.Storage.Search.QueryOptions()
queryOptions.FileTypeFilter.Add(".txt")
queryOptions.FolderDepth = Windows.Storage.Search.FolderDepth.Deep
Dim allFiles As IEnumerable(Of Windows.Storage.StorageFile) = From _file In Await _rootDir.CreateFileQueryWithOptions(queryOptions).GetFilesAsync() Select _file
Dim allDistinctLines As IEnumerable(Of String) = Task.WhenAll(allFiles.Select(Function(_file) Windows.Storage.FileIO.ReadLines.Async(_file).Where(Function(_line) _line.Trim().Length > 0).Distinct()))
Return allDistinctLines
End Function
But the return type isn't the same, it ends up being some strange IList(Of String)(), which doesn't work for me strangely. Maybe there is just a critical misunderstanding there.
Regardless, any help understanding Async file operations is appriciated!
vb.net linq uwp async-await
vb.net linq uwp async-await
asked Nov 8 at 15:10
Tyler Gubala
184313
asked Nov 8 at 15:10
Tyler Gubala
184313
asked Nov 8 at 15:10
Tyler Gubala
184313
asked Nov 8 at 15:10
Tyler Gubala
184313
asked Nov 8 at 15:10
Tyler Gubala
184313
184313
Please check this case reply.
– CoCaIceDew
Nov 9 at 9:35
@CoCalceDew I have seen that post, and it was included in my original question. I need some clarifcation though on how it should be utilized. Any help would be appriciated.
– Tyler Gubala
Nov 12 at 17:21
add a comment |
Please check this case reply.
– CoCaIceDew
Nov 9 at 9:35
@CoCalceDew I have seen that post, and it was included in my original question. I need some clarifcation though on how it should be utilized. Any help would be appriciated.
– Tyler Gubala
Nov 12 at 17:21
Please check this case reply.
– CoCaIceDew
Nov 9 at 9:35
Please check this case reply.
– CoCaIceDew
Nov 9 at 9:35
@CoCalceDew I have seen that post, and it was included in my original question. I need some clarifcation though on how it should be utilized. Any help would be appriciated.
– Tyler Gubala
Nov 12 at 17:21
@CoCalceDew I have seen that post, and it was included in my original question. I need some clarifcation though on how it should be utilized. Any help would be appriciated.
– Tyler Gubala
Nov 12 at 17:21
add a comment |
1 Answer
1
active
oldest
votes
up vote
2
down vote
accepted
As Stephen said in this thread, LINQ has very limited support for async/await. I recommend you use For Each in conjunction with Windows.Storage api instead like the follow:
Async Function getAllLines() As Task(Of IEnumerable(Of String))
Dim _folderPicker As New Windows.Storage.Pickers.FolderPicker()
_folderPicker.FileTypeFilter.Add("*") 'We must add this; it is a known bug
Dim _rootDir As Windows.Storage.StorageFolder = Await _folderPicker.PickSingleFolderAsync()
Dim queryOptions As New Windows.Storage.Search.QueryOptions()
queryOptions.FileTypeFilter.Add(".txt")
queryOptions.FolderDepth = Windows.Storage.Search.FolderDepth.Deep
Dim allDistinctLines As New List(Of String)
For Each file As StorageFile In Await _rootDir.CreateFileQueryWithOptions(queryOptions).GetFilesAsync()
For Each line In Await Windows.Storage.FileIO.ReadLinesAsync(file)
If line.Trim().Length > 0 Then
allDistinctLines.Add(line.Trim())
End If
Next
Next
Return allDistinctLines.Distinct()
End Function
answered Nov 12 at 2:30
Nico Zhu - MSFT
8,8271421
Thanks for the answer. Interestingly, this is almost identical to the currently working code that I ended up producing. I really enjoyed being able to produce a very powerful, singular line of code that enumerated the files to the depth that I specified and retrieved the distinct lines at the same time. Sure, I could write an extension, but this seems like a bit of a loss of expressiveness if that isn't too melodramatic. Still, the enumeration in my testing completed much faster, not that the application is all the better for it; the files I need to enumerate produce an xml of total size < 1MB
– Tyler Gubala
Nov 12 at 17:18
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53210579%2fasync-and-linq-queries-retrieve-all-lines-in-all-files-in-a-directory-tree%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
2
down vote
accepted
As Stephen said in this thread, LINQ has very limited support for async/await. I recommend you use For Each in conjunction with Windows.Storage api instead like the follow:
Async Function getAllLines() As Task(Of IEnumerable(Of String))
Dim _folderPicker As New Windows.Storage.Pickers.FolderPicker()
_folderPicker.FileTypeFilter.Add("*") 'We must add this; it is a known bug
Dim _rootDir As Windows.Storage.StorageFolder = Await _folderPicker.PickSingleFolderAsync()
Dim queryOptions As New Windows.Storage.Search.QueryOptions()
queryOptions.FileTypeFilter.Add(".txt")
queryOptions.FolderDepth = Windows.Storage.Search.FolderDepth.Deep
Dim allDistinctLines As New List(Of String)
For Each file As StorageFile In Await _rootDir.CreateFileQueryWithOptions(queryOptions).GetFilesAsync()
For Each line In Await Windows.Storage.FileIO.ReadLinesAsync(file)
If line.Trim().Length > 0 Then
allDistinctLines.Add(line.Trim())
End If
Next
Next
Return allDistinctLines.Distinct()
End Function
answered Nov 12 at 2:30
Nico Zhu - MSFT
8,8271421
Thanks for the answer. Interestingly, this is almost identical to the currently working code that I ended up producing. I really enjoyed being able to produce a very powerful, singular line of code that enumerated the files to the depth that I specified and retrieved the distinct lines at the same time. Sure, I could write an extension, but this seems like a bit of a loss of expressiveness if that isn't too melodramatic. Still, the enumeration in my testing completed much faster, not that the application is all the better for it; the files I need to enumerate produce an xml of total size < 1MB
– Tyler Gubala
Nov 12 at 17:18
add a comment |
up vote
2
down vote
accepted
As Stephen said in this thread, LINQ has very limited support for async/await. I recommend you use For Each in conjunction with Windows.Storage api instead like the follow:
Async Function getAllLines() As Task(Of IEnumerable(Of String))
Dim _folderPicker As New Windows.Storage.Pickers.FolderPicker()
_folderPicker.FileTypeFilter.Add("*") 'We must add this; it is a known bug
Dim _rootDir As Windows.Storage.StorageFolder = Await _folderPicker.PickSingleFolderAsync()
Dim queryOptions As New Windows.Storage.Search.QueryOptions()
queryOptions.FileTypeFilter.Add(".txt")
queryOptions.FolderDepth = Windows.Storage.Search.FolderDepth.Deep
Dim allDistinctLines As New List(Of String)
For Each file As StorageFile In Await _rootDir.CreateFileQueryWithOptions(queryOptions).GetFilesAsync()
For Each line In Await Windows.Storage.FileIO.ReadLinesAsync(file)
If line.Trim().Length > 0 Then
allDistinctLines.Add(line.Trim())
End If
Next
Next
Return allDistinctLines.Distinct()
End Function
answered Nov 12 at 2:30
Nico Zhu - MSFT
8,8271421
Thanks for the answer. Interestingly, this is almost identical to the currently working code that I ended up producing. I really enjoyed being able to produce a very powerful, singular line of code that enumerated the files to the depth that I specified and retrieved the distinct lines at the same time. Sure, I could write an extension, but this seems like a bit of a loss of expressiveness if that isn't too melodramatic. Still, the enumeration in my testing completed much faster, not that the application is all the better for it; the files I need to enumerate produce an xml of total size < 1MB
– Tyler Gubala
Nov 12 at 17:18
add a comment |
up vote
2
down vote
accepted
up vote
2
down vote
accepted
As Stephen said in this thread, LINQ has very limited support for async/await. I recommend you use For Each in conjunction with Windows.Storage api instead like the follow:
Async Function getAllLines() As Task(Of IEnumerable(Of String))
Dim _folderPicker As New Windows.Storage.Pickers.FolderPicker()
_folderPicker.FileTypeFilter.Add("*") 'We must add this; it is a known bug
Dim _rootDir As Windows.Storage.StorageFolder = Await _folderPicker.PickSingleFolderAsync()
Dim queryOptions As New Windows.Storage.Search.QueryOptions()
queryOptions.FileTypeFilter.Add(".txt")
queryOptions.FolderDepth = Windows.Storage.Search.FolderDepth.Deep
Dim allDistinctLines As New List(Of String)
For Each file As StorageFile In Await _rootDir.CreateFileQueryWithOptions(queryOptions).GetFilesAsync()
For Each line In Await Windows.Storage.FileIO.ReadLinesAsync(file)
If line.Trim().Length > 0 Then
allDistinctLines.Add(line.Trim())
End If
Next
Next
Return allDistinctLines.Distinct()
End Function
answered Nov 12 at 2:30
Nico Zhu - MSFT
8,8271421
As Stephen said in this thread, LINQ has very limited support for async/await. I recommend you use For Each in conjunction with Windows.Storage api instead like the follow:
Async Function getAllLines() As Task(Of IEnumerable(Of String))
Dim _folderPicker As New Windows.Storage.Pickers.FolderPicker()
_folderPicker.FileTypeFilter.Add("*") 'We must add this; it is a known bug
Dim _rootDir As Windows.Storage.StorageFolder = Await _folderPicker.PickSingleFolderAsync()
Dim queryOptions As New Windows.Storage.Search.QueryOptions()
queryOptions.FileTypeFilter.Add(".txt")
queryOptions.FolderDepth = Windows.Storage.Search.FolderDepth.Deep
Dim allDistinctLines As New List(Of String)
For Each file As StorageFile In Await _rootDir.CreateFileQueryWithOptions(queryOptions).GetFilesAsync()
For Each line In Await Windows.Storage.FileIO.ReadLinesAsync(file)
If line.Trim().Length > 0 Then
allDistinctLines.Add(line.Trim())
End If
Next
Next
Return allDistinctLines.Distinct()
End Function
answered Nov 12 at 2:30
Nico Zhu - MSFT
8,8271421
answered Nov 12 at 2:30
Nico Zhu - MSFT
8,8271421
answered Nov 12 at 2:30
Nico Zhu - MSFT
8,8271421
answered Nov 12 at 2:30
Nico Zhu - MSFT
8,8271421
8,8271421
Thanks for the answer. Interestingly, this is almost identical to the currently working code that I ended up producing. I really enjoyed being able to produce a very powerful, singular line of code that enumerated the files to the depth that I specified and retrieved the distinct lines at the same time. Sure, I could write an extension, but this seems like a bit of a loss of expressiveness if that isn't too melodramatic. Still, the enumeration in my testing completed much faster, not that the application is all the better for it; the files I need to enumerate produce an xml of total size < 1MB
– Tyler Gubala
Nov 12 at 17:18
add a comment |
Thanks for the answer. Interestingly, this is almost identical to the currently working code that I ended up producing. I really enjoyed being able to produce a very powerful, singular line of code that enumerated the files to the depth that I specified and retrieved the distinct lines at the same time. Sure, I could write an extension, but this seems like a bit of a loss of expressiveness if that isn't too melodramatic. Still, the enumeration in my testing completed much faster, not that the application is all the better for it; the files I need to enumerate produce an xml of total size < 1MB
– Tyler Gubala
Nov 12 at 17:18
Thanks for the answer. Interestingly, this is almost identical to the currently working code that I ended up producing. I really enjoyed being able to produce a very powerful, singular line of code that enumerated the files to the depth that I specified and retrieved the distinct lines at the same time. Sure, I could write an extension, but this seems like a bit of a loss of expressiveness if that isn't too melodramatic. Still, the enumeration in my testing completed much faster, not that the application is all the better for it; the files I need to enumerate produce an xml of total size < 1MB
– Tyler Gubala
Nov 12 at 17:18
Thanks for the answer. Interestingly, this is almost identical to the currently working code that I ended up producing. I really enjoyed being able to produce a very powerful, singular line of code that enumerated the files to the depth that I specified and retrieved the distinct lines at the same time. Sure, I could write an extension, but this seems like a bit of a loss of expressiveness if that isn't too melodramatic. Still, the enumeration in my testing completed much faster, not that the application is all the better for it; the files I need to enumerate produce an xml of total size < 1MB
– Tyler Gubala
Nov 12 at 17:18
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53210579%2fasync-and-linq-queries-retrieve-all-lines-in-all-files-in-a-directory-tree%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Please check this case reply.
– CoCaIceDew
Nov 9 at 9:35
@CoCalceDew I have seen that post, and it was included in my original question. I need some clarifcation though on how it should be utilized. Any help would be appriciated.
– Tyler Gubala
Nov 12 at 17:21