Myujth

I have some UWP code that I am trying to maintain. Its goal is to coalesce some plaintext files kept in a directory structure into a single XML file.

In the interest of doing this, I previously grab all distinct lines in all files in the dirrectory structure using LINQ:

Imports System.IO

Async Function getAllLines() As IEnumerable(Of String)
 Dim _folderPicker As New Windows.Storage.Pickers.FolderPicker()
 _folderPicker.FileTypeFilter.Add("*") 'We must add this; it is a known bug
 Dim _rootDir As Windows.Storage.StorageFolder = Await _folderPicker.PickSingleFolderAsync()

 Dim allDistinctLines As IEnumerable(Of String) = From _file In Directory.GetFiles(_rootDir.Name, "*.txt", SearchOption.AllDirectories) From line In File.ReadAllLines(_file) Where line.Trim().Length > 0 Select line.Trim() Distinct

 Return allDistinctLines
End Function

Since late October, however, it seems some things have changed regarding file perrmissions (Source). I came back to the code after about a month of it working to find it throwing an exception saying that the folder that I was previously able to write to was not accessable due to permissions. The folder existed and I grabbed the folder from a picker so I thought that I was good. Previously, as long as I had grabbed a File or Folder using a picker, I could still use a System.IO function to enumerate, read from or write to said folder or directory.

Now, it seems as though you must go though the proper UWP faculties, which is fine, but I am unsure how to rewrite my LINQ query as the UWP methods are asynchronous.

Here is what I have tried:

Async Function getAllLines() As IEnumerable(Of String)
 Dim _folderPicker As New Windows.Storage.Pickers.FolderPicker()
 _folderPicker.FileTypeFilter.Add("*") 'We must add this; it is a known bug
 Dim _rootDir As Windows.Storage.StorageFolder = Await _folderPicker.PickSingleFolderAsync()

 Dim queryOptions As New Windows.Storage.Search.QueryOptions()
 queryOptions.FileTypeFilter.Add(".txt")
 queryOptions.FolderDepth = Windows.Storage.Search.FolderDepth.Deep

 Dim allDistinctLines As IEnumerable(Of String) = From _file In Await _rootDir.CreateFileQueryWithOptions(queryOptions).GetFilesAsync() From line In Await Windows.Storage.FileIO.ReadLinesAsync(_file) Where line.Trim().Length > 0 Select line.Trim() Distinct

 Return allDistinctLines
End Function

In the above code, I am at least able to enumerate the folder if I comment out the second await in the LINQ. The issue is that with the second Await in the LINQ query, I cannot compile as

'Await' may only be used in a query expression within the first collection expression of the initial 'From' clause or within the collection expression of a 'Join' clause.

So I read up on Join clauses, which seem to equate disparate data, which isn't really what I am looking for.

I read up on the error in general and it seems like the LINQ queries and Async Functions only have limited functionality. Honestly, I don't know enough about LINQ to really make that claim, but that was my feeling.

I found this answer, which uses Task.WhenAll() to facilitate the LINQ and Async Functions usage, but I can't wrap my head around exactly how it would be put into practice.

How to await a method in a Linq query

So I tried:

Async Function getAllLines() As IEnumerable(Of String)
 Dim _folderPicker As New Windows.Storage.Pickers.FolderPicker()
 _folderPicker.FileTypeFilter.Add("*") 'We must add this; it is a known bug
 Dim _rootDir As Windows.Storage.StorageFolder = Await _folderPicker.PickSingleFolderAsync()

 Dim queryOptions As New Windows.Storage.Search.QueryOptions()
 queryOptions.FileTypeFilter.Add(".txt")
 queryOptions.FolderDepth = Windows.Storage.Search.FolderDepth.Deep

 Dim allFiles As IEnumerable(Of Windows.Storage.StorageFile) = From _file In Await _rootDir.CreateFileQueryWithOptions(queryOptions).GetFilesAsync() Select _file

 Dim allDistinctLines As IEnumerable(Of String) = Task.WhenAll(allFiles.Select(Function(_file) Windows.Storage.FileIO.ReadLines.Async(_file).Where(Function(_line) _line.Trim().Length > 0).Distinct()))

 Return allDistinctLines
End Function

But the return type isn't the same, it ends up being some strange IList(Of String)(), which doesn't work for me strangely. Maybe there is just a critical misunderstanding there.

Regardless, any help understanding Async file operations is appriciated!

As Stephen said in this thread, LINQ has very limited support for async/await. I recommend you use For Each in conjunction with Windows.Storage api instead like the follow:

Async Function getAllLines() As Task(Of IEnumerable(Of String))
 Dim _folderPicker As New Windows.Storage.Pickers.FolderPicker()
 _folderPicker.FileTypeFilter.Add("*") 'We must add this; it is a known bug
 Dim _rootDir As Windows.Storage.StorageFolder = Await _folderPicker.PickSingleFolderAsync()

 Dim queryOptions As New Windows.Storage.Search.QueryOptions()
 queryOptions.FileTypeFilter.Add(".txt")
 queryOptions.FolderDepth = Windows.Storage.Search.FolderDepth.Deep

 Dim allDistinctLines As New List(Of String)

 For Each file As StorageFile In Await _rootDir.CreateFileQueryWithOptions(queryOptions).GetFilesAsync()
 For Each line In Await Windows.Storage.FileIO.ReadLinesAsync(file)
 If line.Trim().Length > 0 Then
 allDistinctLines.Add(line.Trim())
 End If
 Next
 Next

 Return allDistinctLines.Distinct()
End Function