Myujth

I'm getting a failed builds due to the Visual Studio Test step failing in my Build Pipeline.

I have a simple .NET Core v2.1 class library and associated MS Test library. I have 2 steps in my pipeline:

1. A .NET Core build step, and


2. A Visual Studio Test step

Both projects target the netcoreapp2.1 framework, see below for .csproj files.

I have tried enabling and disabling the Code coverage option, but this make no difference.

Thanks in advance,

Kaine

.csproj files

Main Project

<Project Sdk="Microsoft.NET.Sdk">

 <PropertyGroup>
 <TargetFramework>netcoreapp2.1</TargetFramework>
 <AssemblyName>Trustmarque.Core.ExtensionMethods</AssemblyName>
 <RootNamespace>Trustmarque.Core.ExtensionMethods</RootNamespace>
 <GeneratePackageOnBuild>true</GeneratePackageOnBuild>
 <Description>Generalised class extension methods build on the .NET Core.</Description>
 </PropertyGroup>

 <ItemGroup>
 <PackageReference Include="Newtonsoft.Json" Version="11.0.2" />
 </ItemGroup>

</Project>

Test Project

<Project Sdk="Microsoft.NET.Sdk">

 <PropertyGroup>
 <TargetFramework>netcoreapp2.1</TargetFramework>

 <IsPackable>false</IsPackable>

 <AssemblyName>Trustmarque.Core.ExtensionMethods.Tests</AssemblyName>

 <RootNamespace>Trustmarque.Core.ExtensionMethods.Tests</RootNamespace>
 </PropertyGroup>

 <ItemGroup>
 <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.9.0" />
 <PackageReference Include="MSTest.TestAdapter" Version="1.3.2" />
 <PackageReference Include="MSTest.TestFramework" Version="1.3.2" />
 </ItemGroup>

 <ItemGroup>
 <ProjectReference Include="..ExtensionMethodsExtensionMethods.csproj" />
 </ItemGroup>

</Project>

YAML

queue:
 name: Hosted VS2017
 demands: vstest

steps:
- task: DotNetCoreCLI@2
 displayName: '.NET Core Build'


- task: VSTest@2
 displayName: 'VS Unit Test'
 inputs:
 testFiltercriteria: /diag

 codeCoverageEnabled: true

 otherConsoleOptions: '/framework:".NETCoreApp,Version=2.1"'

Test Output Log

2018-11-14T09:33:56.0414482Z ##[section]Starting: VS Unit Test
2018-11-14T09:33:56.0420823Z ==============================================================================
2018-11-14T09:33:56.0420887Z Task : Visual Studio Test
2018-11-14T09:33:56.0420941Z Description : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test (VsTest) runner. Test frameworks that have a Visual Studio test adapter such as MsTest, xUnit, NUnit, Chutzpah (for JavaScript tests using QUnit, Mocha and Jasmine), etc. can be run. Tests can be distributed on multiple agents using this task (version 2).
2018-11-14T09:33:56.0420983Z Version : 2.142.12
2018-11-14T09:33:56.0421033Z Author : Microsoft Corporation
2018-11-14T09:33:56.0421089Z Help : [More information](https://go.microsoft.com/fwlink/?LinkId=835764)
2018-11-14T09:33:56.0421126Z ==============================================================================
2018-11-14T09:33:56.9036862Z SystemVssConnection exists true
2018-11-14T09:33:57.0359241Z SystemVssConnection exists true
2018-11-14T09:33:57.1181619Z Running tests using vstest.console.exe runner.
2018-11-14T09:33:57.1181728Z ======================================================
2018-11-14T09:33:57.1182048Z Test selector : Test assemblies
2018-11-14T09:33:57.1182141Z Test filter criteria : null
2018-11-14T09:33:57.1182212Z Search folder : D:a1s
2018-11-14T09:33:57.1182743Z VisualStudio version selected for test execution : latest
2018-11-14T09:33:57.8312901Z Run in parallel : false
2018-11-14T09:33:57.8314015Z Run in isolation : false
2018-11-14T09:33:57.8315114Z Path to custom adapters : null
2018-11-14T09:33:57.8319882Z Other console options : /framework:".NETCoreApp,Version=2.1"
2018-11-14T09:33:57.8320427Z Code coverage enabled : true
2018-11-14T09:33:57.8320689Z Diagnostics enabled : false
2018-11-14T09:33:57.8322155Z SystemVssConnection exists true
2018-11-14T09:33:57.8348561Z Run the tests locally using vstest.console.exe
2018-11-14T09:33:57.8348848Z ========================================================
2018-11-14T09:33:57.8349439Z Test selector : Test assemblies
2018-11-14T09:33:57.8352816Z Test assemblies : ***test*.dll,!***TestAdapter.dll,!**obj**
2018-11-14T09:33:57.8353669Z Test filter criteria : null
2018-11-14T09:33:57.8360790Z Search folder : D:a1s
2018-11-14T09:33:57.8361971Z Run settings file : D:a1s
2018-11-14T09:33:57.8369736Z Run in parallel : false
2018-11-14T09:33:57.8370326Z Run in isolation : false
2018-11-14T09:33:57.8392182Z Path to custom adapters : null
2018-11-14T09:33:57.8393171Z Other console options : /framework:".NETCoreApp,Version=2.1"
2018-11-14T09:33:57.8395168Z Code coverage enabled : true
2018-11-14T09:33:57.8396140Z Diagnostics enabled : false
2018-11-14T09:33:57.8398717Z Rerun failed tests: false
2018-11-14T09:33:57.8401805Z VisualStudio version selected for test execution : latest
2018-11-14T09:33:58.9933609Z ========================================================
2018-11-14T09:33:58.9973631Z ======================================================
2018-11-14T09:33:59.0613609Z [command]"C:Program Files (x86)Microsoft Visual Studio2017EnterpriseCommon7IDECommonExtensionsMicrosoftTestWindowvstest.console.exe" @D:a_temp69e3bc61-e7f0-11e8-8572-4338aa006dd6.txt
2018-11-14T09:34:06.3128046Z Microsoft (R) Test Execution Command Line Tool Version 15.8.0
2018-11-14T09:34:06.3130539Z Copyright (c) Microsoft Corporation. All rights reserved.
2018-11-14T09:34:06.3131113Z 
2018-11-14T09:34:06.6145727Z vstest.console.exe 
2018-11-14T09:34:06.6146432Z "D:a1sExtensionMethods.TestsbinDebugnetcoreapp2.1Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.Interface.dll"
2018-11-14T09:34:06.6147591Z "D:a1sExtensionMethods.TestsbinDebugnetcoreapp2.1Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.dll"
2018-11-14T09:34:06.6147793Z "D:a1sExtensionMethods.TestsbinDebugnetcoreapp2.1Trustmarque.Core.ExtensionMethods.Tests.dll"
2018-11-14T09:34:06.6149501Z /EnableCodeCoverage
2018-11-14T09:34:06.6149657Z /logger:"trx"
2018-11-14T09:34:06.6149783Z /TestAdapterPath:"D:a1s"
2018-11-14T09:34:06.6149907Z /framework:".NETCoreApp,Version=2.1"
2018-11-14T09:34:12.8077290Z Starting test execution, please wait...
2018-11-14T09:34:14.8437128Z Test run will use DLL(s) built for framework .NETCoreApp,Version=v2.1 and platform X86. Following DLL(s) do not match framework/platform settings.
2018-11-14T09:34:14.8437793Z Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.Interface.dll is built for Framework 4.5 and Platform AnyCPU.
2018-11-14T09:34:14.8438215Z Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.dll is built for Framework 1.5 and Platform AnyCPU.
2018-11-14T09:34:14.8438617Z Go to http://go.microsoft.com/fwlink/?LinkID=236877&clcid=0x409 for more details on managing these settings.
2018-11-14T09:34:14.8438786Z 
2018-11-14T09:34:23.5616980Z M i c r o s o f t ( R ) C o v e r a g e C o l l e c t i o n T o o l V e r s i o n 1 5 . 0 . 3 0 3 1 9 . 0 
2018-11-14T09:34:23.5617265Z 
2018-11-14T09:34:23.5617351Z 
2018-11-14T09:34:23.5617457Z C o p y r i g h t ( c ) M i c r o s o f t C o r p o r a t i o n . A l l r i g h t s r e s e r v e d . 
2018-11-14T09:34:23.5617498Z 
2018-11-14T09:34:23.5617554Z 
2018-11-14T09:34:23.5617591Z 
2018-11-14T09:34:23.5617627Z 
2018-11-14T09:34:24.4903850Z 2.8847
2018-11-14T09:34:25.4504736Z System.IO.FileNotFoundException: Unable to find tests for D:a1sExtensionMethods.TestsbinDebugnetcoreapp2.1Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.Interface.dll. Make sure test project has a nuget reference of package "Microsoft.NET.Test.Sdk" and framework version settings are appropriate. Rerun with /diag option to diagnose further.
2018-11-14T09:34:25.4504927Z at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DotnetTestHostManager.GetTestHostProcessStartInfo(IEnumerable`1 sources, IDictionary`2 environmentVariables, TestRunnerConnectionInfo connectionInfo)
2018-11-14T09:34:25.4505025Z at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources)
2018-11-14T09:34:25.4505077Z at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyExecutionManager.StartTestRun(TestRunCriteria testRunCriteria, ITestRunEventsHandler eventHandler)
2018-11-14T09:34:25.6678123Z M i c r o s o f t ( R ) C o v e r a g e C o l l e c t i o n T o o l V e r s i o n 1 5 . 0 . 3 0 3 1 9 . 0 
2018-11-14T09:34:25.6678286Z 
2018-11-14T09:34:25.6678334Z 
2018-11-14T09:34:25.6678492Z C o p y r i g h t ( c ) M i c r o s o f t C o r p o r a t i o n . A l l r i g h t s r e s e r v e d . 
2018-11-14T09:34:25.6678566Z 
2018-11-14T09:34:25.6678623Z 
2018-11-14T09:34:25.6678664Z 
2018-11-14T09:34:25.6678700Z 
2018-11-14T09:34:26.8351769Z M i c r o s o f t ( R ) C o v e r a g e C o l l e c t i o n T o o l V e r s i o n 1 5 . 0 . 3 0 3 1 9 . 0 
2018-11-14T09:34:26.8351940Z 
2018-11-14T09:34:26.8352006Z 
2018-11-14T09:34:26.8352092Z C o p y r i g h t ( c ) M i c r o s o f t C o r p o r a t i o n . A l l r i g h t s r e s e r v e d . 
2018-11-14T09:34:26.8352125Z 
2018-11-14T09:34:26.8352155Z 
2018-11-14T09:34:26.8352199Z 
2018-11-14T09:34:26.8352227Z 
2018-11-14T09:34:26.8788931Z 2.3838
2018-11-14T09:34:26.8898822Z System.IO.FileNotFoundException: Unable to find tests for D:a1sExtensionMethods.TestsbinDebugnetcoreapp2.1Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.dll. Make sure test project has a nuget reference of package "Microsoft.NET.Test.Sdk" and framework version settings are appropriate. Rerun with /diag option to diagnose further.
2018-11-14T09:34:26.8899122Z at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DotnetTestHostManager.GetTestHostProcessStartInfo(IEnumerable`1 sources, IDictionary`2 environmentVariables, TestRunnerConnectionInfo connectionInfo)
2018-11-14T09:34:26.8899942Z at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources)
2018-11-14T09:34:26.8900007Z at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyExecutionManager.StartTestRun(TestRunCriteria testRunCriteria, ITestRunEventsHandler eventHandler)
2018-11-14T09:34:26.9247283Z M i c r o s o f t ( R ) C o v e r a g e C o l l e c t i o n T o o l V e r s i o n 1 5 . 0 . 3 0 3 1 9 . 0 
2018-11-14T09:34:26.9247370Z 
2018-11-14T09:34:26.9247428Z 
2018-11-14T09:34:26.9247520Z C o p y r i g h t ( c ) M i c r o s o f t C o r p o r a t i o n . A l l r i g h t s r e s e r v e d . 
2018-11-14T09:34:26.9247570Z 
2018-11-14T09:34:26.9247619Z 
2018-11-14T09:34:26.9247650Z 
2018-11-14T09:34:26.9247681Z 
2018-11-14T09:34:27.4568128Z M i c r o s o f t ( R ) C o v e r a g e C o l l e c t i o n T o o l V e r s i o n 1 5 . 0 . 3 0 3 1 9 . 0 
2018-11-14T09:34:27.4568259Z 
2018-11-14T09:34:27.4568335Z 
2018-11-14T09:34:27.4568442Z C o p y r i g h t ( c ) M i c r o s o f t C o r p o r a t i o n . A l l r i g h t s r e s e r v e d . 
2018-11-14T09:34:27.4568504Z 
2018-11-14T09:34:27.4568543Z 
2018-11-14T09:34:27.4568580Z 
2018-11-14T09:34:27.4568631Z 
2018-11-14T09:34:27.5016363Z 3.0114
2018-11-14T09:34:33.4517174Z M i c r o s o f t ( R ) C o v e r a g e C o l l e c t i o n T o o l V e r s i o n 1 5 . 0 . 3 0 3 1 9 . 0 
2018-11-14T09:34:33.4517363Z 
2018-11-14T09:34:33.4517409Z 
2018-11-14T09:34:33.4517517Z C o p y r i g h t ( c ) M i c r o s o f t C o r p o r a t i o n . A l l r i g h t s r e s e r v e d . 
2018-11-14T09:34:33.4517603Z 
2018-11-14T09:34:33.4517641Z 
2018-11-14T09:34:33.4517678Z 
2018-11-14T09:34:33.4517715Z 
2018-11-14T09:34:33.7048135Z Passed Timestamp_DateTime_CurrentDateTime_FormatAndValuesMatch
2018-11-14T09:34:33.7048569Z Passed ToJson_Object_ReturnsStringRepresentation
2018-11-14T09:34:33.7048660Z Passed ToJsonString_Object_ReturnsStringRepresentationIncludingObjectType
2018-11-14T09:34:33.7049036Z Passed IsNullOrEmpty_String_EmptyAndNullValues
2018-11-14T09:34:33.7049135Z Passed IsNullOrEmpty_String_NonEmptyOrNullValues
2018-11-14T09:34:33.7049190Z Passed Parse_String_ValidTypeParses_CorrectTypes
2018-11-14T09:34:33.7049257Z Passed Parse_String_ValidTypeParses_DefaultTypes
2018-11-14T09:34:33.7049297Z Passed FromJson_String_ValidObjects
2018-11-14T09:34:33.7049338Z Passed FromJson_String_InvalidObjects
2018-11-14T09:34:33.7049402Z Passed IsValidEmailAddress_String_ValidEmailAddresses_ReturnTrue
2018-11-14T09:34:33.7049465Z Passed IsValidEmailAddress_String_InvalidEmailAddresses_ReturnFalse
2018-11-14T09:34:35.8172642Z Results File: D:a1sTestResultsVssAdministrator_factoryvm-az552_2018-11-14_09_34_33.trx
2018-11-14T09:34:35.8203749Z 
2018-11-14T09:34:35.8210373Z Attachments:
2018-11-14T09:34:35.8210758Z D:a1sTestResults6c92c0b5-2e35-48af-867a-2a4575306a68VssAdministrator_factoryvm-az552 2018-11-14 09_34_22.coverage
2018-11-14T09:34:35.8211079Z 
2018-11-14T09:34:35.8211397Z Total tests: Unknown. Passed: 11. Failed: 0. Skipped: 0.
2018-11-14T09:34:35.8211670Z Test Run Aborted.
2018-11-14T09:34:35.8211939Z Test execution time: 10.7395 Seconds
2018-11-14T09:34:36.0240507Z ##[warning]Vstest failed with error. Check logs for failures. There might be failed tests.
2018-11-14T09:34:36.0769776Z ##[error]Error: C:Program Files (x86)Microsoft Visual Studio2017EnterpriseCommon7IDECommonExtensionsMicrosoftTestWindowvstest.console.exe failed with return code: 1
2018-11-14T09:34:36.3170630Z ##[error]VsTest task failed.
2018-11-14T09:34:36.3566464Z ##[section]Async Command Start: Publish test results
2018-11-14T09:34:36.4181634Z Publishing test results to test run '6'
2018-11-14T09:34:36.4181725Z Test results remaining: 11. Test run id: 6
2018-11-14T09:34:36.9962474Z Published Test Run : https://dev.azure.com/TrustmarqueIT/Test_2018-10-30/_TestManagement/Runs#runId=6&_a=runCharts
2018-11-14T09:34:36.9971952Z ##[section]Async Command End: Publish test results
2018-11-14T09:34:36.9972803Z ##[section]Finishing: VS Unit Test

Thanks @Gurpreet for prompting me to document my solution.

It's been quite a while, so my memory of the steps is a bit shaky, but I believe that my mistake was thinking that, on the first screen after selecting a source repository, I assumed the templates listed were all that was available and that Empty job was the same as the YAML template. I'd incorrectly assumed that Empty job meant that I'd have to edit the YAML myself.

Jonathan Twite gave me some great pointers but I didn't quite grasp what he was suggesting at the time.

The Solution

Anyway, the solution was to select either Empty job or scroll to the bottom and select Empty pipeline, why they're referred to by different names is a mystery.

After selecting the empty template, the next screen provides you with a default Agent job. Clicking the plus, you can add a variety of different tasks. This was part of my original mistake, I confused the tasks with the templates on the previous screen.

I added the .NET Core task.

I changed the Display name and the Command, as well as making sure that the Version was correct. Interestingly, the Command has a number of interesting options, three of which that I found useful are:

• restore


• build


• test

So, rather than adding a specific Test task, you add the .NET Core task and configure it to do the test bit. This was a bit confusing for me, I didn't realise that the .NET Core task was capable of doing the testing as well.

In my case, I also started testing the ability to deploy NuGet packages, but that's a whole other can of worms.

I hope that this has helped, and wish you all good luck with your projects.

Kaine

I have solved my problem by revising my filter to find test assemblies. when i was messing around with my test step then i had set the filter to "/**/UnitTests.dll" and i did not realise that it was also matching UnitTests.dll in the "obj" folder and that folder does not have .deps.json files. from the logs i noticed that it was running tests twice (once for each matching dll) so then i ignored the obj folder by reverting back my filter to exclude obj folder. So i guess two things to take care in such cases, 1. you must have "Microsoft.NET.Test.Sdk" installed on your test project and 2. your filter does not include or exclude any unwanted dlls which you (dont want / want to) be there