unittest-msbuild-nunit-vstest

name: unittest-msbuild-nunit-vstest description: > ソリューション（.sln/.slnx）のユニットテスト（NUnit）を実行する。 C# + C++/CLI + C++ 混在のため MSBuild(vswhere) で先にビルドし、NUnitテストDLLのみを vstest / dotnet vstest で実行する。 license: CC-BY-4.0

Goal

Run NUnit tests reliably in a solution that contains C# + C++/CLI + C++ projects, avoiding false failures from non-test projects.

MUST rules (critical)

• MUST run build with msbuild from PowerShell, locating MSBuild.exe via vswhere and storing it in $msbuild.
• MUST NOT run dotnet test against the solution/root, because mixed-language projects can trigger build/evaluation failures and confuse result interpretation.
• MUST discover NUnit test projects, then run tests ONLY for the resulting test assemblies (.dll).
• MUST determine pass/fail ONLY by the exit code of the invoked tools:
  • Build: exit code 0 => build success; non-zero => stop (do not proceed to tests).
  • Test run: exit code 0 => success; non-zero => failure.
  • Do NOT judge by the presence of the word "error" in output logs.

Step 1: Build (MSBuild via vswhere)

Use this exact pattern:

$vswhere = "${env:ProgramFiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe"

$msbuild = & $vswhere -latest -products * -requires Microsoft.Component.MSBuild `
  -find "MSBuild\**\Bin\MSBuild.exe"

if (-not $msbuild) { throw "MSBuild.exe not found. Install VS/BuildTools with MSBuild." }

# Use full path to .sln/.slnx
& $msbuild "<FULL_PATH_TO_SOLUTION.slnx>" /restore /m /p:Configuration=Debug /p:Platform=x64
if ($LASTEXITCODE -ne 0) { throw "Build failed. Stop here." }

Step 2: Discover NUnit test projects

Discovery policy (prefer A, fallback B):

A) Prefer: identify C# test projects by inspecting *.csproj:

• true OR
• PackageReference includes "Microsoft.NET.Test.Sdk" AND ("NUnit" OR "NUnit3TestAdapter")

B) Fallback: use a fixed project-name prefix convention (e.g., "Tests." or "") and treat matching outputs as test assemblies.

Example PowerShell outline:

$repoRoot = (Get-Location).Path
$csprojs = Get-ChildItem $repoRoot -Recurse -Filter *.csproj

function Is-NUnitTestProject($csprojPath) {
  [xml]$xml = Get-Content $csprojPath
  $isTest = $xml.Project.PropertyGroup.IsTestProject -contains "true"
  $pkg = @($xml.Project.ItemGroup.PackageReference) | ForEach-Object { $_.Include }
  $hasSdk = $pkg -contains "Microsoft.NET.Test.Sdk"
  $hasNUnit = ($pkg -contains "NUnit") -or ($pkg -contains "NUnit3TestAdapter")
  return $isTest -or ($hasSdk -and $hasNUnit)
}

$testProjects = $csprojs | Where-Object { Is-NUnitTestProject $_.FullName }
if (-not $testProjects) { throw "No NUnit test projects found." }

Step 3: Resolve built test assemblies (DLLs)

For each discovered test project:

• Determine assembly name: if present, else project filename.

• Find built DLL under common MSBuild output layouts:

  • \bin**.dll
  • \bin**.dll

Example outline:

$configuration = "Debug"
$platform = "x64"

function Get-AssemblyName($csprojPath) {
  [xml]$xml = Get-Content $csprojPath
  $name = $xml.Project.PropertyGroup.AssemblyName | Select-Object -First 1
  if ($name) { return $name }
  return [System.IO.Path]::GetFileNameWithoutExtension($csprojPath)
}

$testDlls = @()

foreach ($p in $testProjects) {
  $projDir = Split-Path $p.FullName
  $asm = Get-AssemblyName $p.FullName

  $candidates = @(
    Get-ChildItem (Join-Path $projDir "bin\$configuration") -Recurse -Filter "$asm.dll" -ErrorAction SilentlyContinue
    Get-ChildItem (Join-Path $projDir "bin\$platform\$configuration") -Recurse -Filter "$asm.dll" -ErrorAction SilentlyContinue
  ) | Where-Object { $_ } | Select-Object -ExpandProperty FullName -Unique

  if (-not $candidates) { throw "Built test DLL not found for $($p.FullName). Ensure build produced outputs." }

  # Prefer a dll that has a sibling *.runtimeconfig.json (common for .NET test assemblies)
  $preferred = $candidates | Where-Object { Test-Path ([System.IO.Path]::ChangeExtension($_, "runtimeconfig.json")) } | Select-Object -First 1
  $testDlls += ($preferred ?? ($candidates | Select-Object -First 1))
}

$testDlls = $testDlls | Select-Object -Unique

Step 4: Run tests using VSTest (preferred) or dotnet vstest (fallback)

Preferred runner: vstest.console.exe if available (works well with adapters).

Locate vstest.console.exe via vswhere:

$vstest = & $vswhere -latest -products * -find "**\vstest.console.exe" | Select-Object -First 1

Adapter handling:

• If tests use NUnit via VSTest, ensure the NUnit3TestAdapter is discoverable.
• If discovery fails, pass an explicit adapter path.

Run (preferred):

$resultsDir = Join-Path $repoRoot "TestResults"
New-Item -ItemType Directory -Force -Path $resultsDir | Out-Null

foreach ($dll in $testDlls) {
  if ($vstest) {
    & $vstest $dll /Platform:$platform /Logger:trx /ResultsDirectory:$resultsDir
    if ($LASTEXITCODE -ne 0) { throw "Tests failed for $dll" }
  } else {
    dotnet vstest $dll --ResultsDirectory:$resultsDir
    if ($LASTEXITCODE -ne 0) { throw "Tests failed for $dll" }
  }
}

"ALL TESTS PASSED"

Reporting

• Report the list of test DLLs executed.

• Report the location of TRX/results if generated.

• Final verdict MUST follow exit codes:

  • build exit code != 0 => BUILD FAILED
  • any test run exit code != 0 => TESTS FAILED
  • otherwise => PASS