Not working with .NET core .csproj files with Sdks\Microsoft.NET.Sdk\Sdk\Sdk.props not found error

[dlech]

I have made a sample repository for reproducing the problem available at https://github.com/dlech/docfx_project.

I am using DocFX v2.18.4 installed from the chocolatey package on Windows 10 with VS 2017 Preview (15.3 preview 2).

This is basically the default docfx init with a C# project added to the src/ directory (dotnet new classlib -o src/) using dotnet2.0 preview tooling.

When running docfx, I get the unhelpful warning (see #1686):

[17-06-09 03:41:02.022]Warning:No metadata is generated for Core.

Running more than once results in an extra warning:

[17-06-09 04:19:08.622]Warning:Cache for C:/Users/david/Documents/GitHub/Dandy.Core/Dandy.Core.sln in C:\Users\david\Documents\GitHub\Dandy.Core\obj\xdoc\cache\final\-262745601 is corrupted
[17-06-09 04:19:08.622]Warning:No metadata is generated for Core.

After cherry-picking #1738, I get the additional message:

[17-06-09 04:23:49.102]Warning:Workspace failed with: [Failure] Msbuild failed when processing the file 'C:\Users\david\Documents\GitHub\Dandy.Core\src\Core.csproj' with message: The imported project "C:\users\david\Documents\GitHub\docfx\src\docfx\bin\Debug\net46\Sdks\Microsoft.NET.Sdk\Sdk\Sdk.props" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.  C:\Users\david\Documents\GitHub\Dandy.Core\src\Core.csproj

So, it appears that DocFX is now expecting to find the .NET SDK in a folder relative to the docfx.exe.

[vicancy]

dotnet/msbuild#1697 talks about the issue, current workaround is to set env.MSBuildSDKsPath to the right place, e.g. C:\Program Files\dotnet\sdk\1.0.4\Sdks. If it is net core 2.0, also try #1752 (comment)

[cgillum]

I'm facing the same problem. I tried the suggested workaround with docfx 2.19.1 and .NET Core 2.0 but it fails trying to find the next import:

set MSBuildExtensionsPath=C:\Program Files\dotnet\sdk\2.0.0-preview1-005977\Sdks

New output:

[17-06-27 04:51:03.896]Warning:[ExtractMetadata]Workspace failed with: [Failure]
  Msbuild failed when processing the file 'C:\Code\GitHub\project\project.csproj' with message: 
  The imported project "C:\Program Files\dotnet\sdk\2.0.0-preview1-005977\Sdks\2.0\Microsoft.Common.props" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.
  C:\Program Files\dotnet\sdk\2.0.0-preview1-005977\Sdks\Microsoft.NET.Sdk\Sdk\Sdk.props

[cgillum]

I should note that I don't have any problem with the new *.csproj file format if I have VS2017 installed on the same machine. It only appears to be a problem when on a machine with just the .NET Core tools installed.

[Dids]

Workaround doesn't seem to work on macOS either.

[dealdiane]

Works on my dev machine where I have Visual Studio installed and .NET Core SDK. However, in the build server where I only have the build tools installed and .NET Core SDK, I get the same error.

Setting the MSBuildExtensionsPath will fix the first error but will come up with a new one: The imported project "C:\Program Files (x86)\MSBuild\2.0\Microsoft.Common.props" was not found. Fixing that by copying folders or setting the default msbuild ToolVersion only makes it worse.

Any other workaround for now?

[vicancy]

Hi @dealdiane , according to dotnet/msbuild#1697 . there is a gap between msbuild tool and .NET core SDK build, so you have to install Visual Studio 2017 (not Build Tools but the full VS) on your build server as current workaround....

[dealdiane]

Thanks @vicancy. Unfortunately, the server is running 2012 (non R2). Visual Studio 2017 will only install on R2 and above.

[Sunzhuokai]

maybe set the ToolsVersion in csproj file can work
<Project Sdk="Microsoft.NET.Sdk" ToolsVersion="14.0">
the version value must be available

[christophwille]

docfx 2.22.3.0 latest via choco. On a machine with 15.3 RTW on it - tried the "set" trick mentioned here.

[17-08-17 12:35:02.800]Warning:[ExtractMetadata]Workspace failed with: [Failure] Msbuild failed when processing the file 'D:\GitWorkspace\SharpZipLib\src\ICSharpCode.SharpZipLib\ICSharpCode.SharpZipLib.csproj' with message: The imported project "C:\ProgramData\chocolatey\lib\docfx\tools\Sdks\Microsoft.NET.Sdk\Sdk\Sdk.props" was not found. Confirm that the path in the declaration is correct, and that the file exists on disk. D:\GitWorkspace\SharpZipLib\src\ICSharpCode.SharpZipLib\ICSharpCode.SharpZipLib.csproj
[17-08-17 12:35:02.890]Warning:[ExtractMetadata]Project 'D:\GitWorkspace\SharpZipLib\src\ICSharpCode.SharpZipLib\ICSharpCode.SharpZipLib.csproj' does not contain any documents.
[17-08-17 12:35:02.921]Warning:[ExtractMetadata]Cache for D:/GitWorkspace/SharpZipLib/src/ICSharpCode.SharpZipLib/ICSharpCode.SharpZipLib.csproj in D:\GitWorkspace\SharpZipLib\src\ICSharpCode.SharpZipLib\obj\xdoc\cache\final\154310149 is corrupted

[vicancy]

Hi @christophwille Could you try invoke the command from Developer Command Prompt for VS2017 and have a retry?

[christophwille]

D:\GitWorkspace\SharpZipLib\docs\help>docfx metadata
[17-08-18 05:09:05.269]Info:Config file docfx.json found, start generating metadata...
[17-08-18 05:09:16.144]Info:Completed in 10827.7402 milliseconds

Build succeeded.
0 Warning(s)
0 Error(s)

---

Yes, that seems to work (I didn't do any of the set magic beforehand)

[sandersaares]

I confirm using VS developer command prompt is a workaround for this issue. I expect DocFX to be able to determine any relevant paths on its own, though, without having to do this.

[jskeet]

This is affecting AppVeyor builds now - the VS2017 image for AppVeyor contains the .NET Core 2.0 SDK (by default) and we're running into this issue. I haven't worked out how to fix it yet - obviously we don't have much control over exactly what's installed on AppVeyor.

Hopefully just adding an environment variable will help, but I need to work out which. (That's assuming the relevant files are available somewhere...)

[jskeet]

Setting MSBuildSDKsPath moves the problem on one step on AppVeyor. I then get:

The imported project "C:\Program Files (x86)\MSBuild\2.0\Microsoft.Common.props"

Setting MSBuildExtensionsPath fixes the first part of that part, but I've no idea why the MSBuild workspace resolver thinks that MSBuildToolsVersion is 2.0. There's no equivalent to passing in tv on the msbuild command line as far as I can see, and setting the ToolsVersion attribute on the project doesn't seem to make any difference either.

[vicancy]

Hi @jskeet I doubt the issue is related to https://docs.microsoft.com/en-us/visualstudio/msbuild/what-s-new-in-msbuild-15-0 that toolsversion is no longer in registry key. Try the following workaround

1. set MSBUILD_EXE_PATH=C:\Program Files\dotnet\sdk\2.0.0\MSBuild.dll
2. Copy attached txt to the same folder as MSBuild.dll and rename it to MSBuild.dll.config, this is the msbuild.exe.config file I copied from a machine with VS2017 installed in which defines msbuildToolsets
   MSBuild.dll.config.txt

It works in my clean machine with only netcore 2.0 installed, please have a try

[jskeet]

Thanks @vicancy - when you say "the same folder" do you mean the folder where I run docfx?

[vicancy]

@jskeet the same folder as MSBuild.dll,

[jskeet]

Hmm... I doubt that I'll be able to write to that folder on a CI machine, but I'll give it a go.

[sandersaares]

set VisualStudioVersion=15.0
set VSINSTALLDIR=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\

These two environment variables appear to resolve this issue - at least I get no more warnings in the output.

[vicancy]

Hey @dealdiane could you try #1752 (comment) as a workaround

[alanedwardes]

Hi, has there been any progress on this issue? I'd like to avoid having to install Visual Studio on the build slave.

[vicancy]

Hi @alanedwardes, if it is netcore project, workaround #1752 (comment) should work, no need to install VS.

[rschili]

So, there is a pretty good understanding now, what the issue is, but there is no good workaround, yet. Just like @jskeet I cannot put dll.config files into Program Files on a build machine, I do not have access to that folder. Visual Studio is also not installed on those machines, just the NET Core SDK.

Currently I am avoiding the issue by not including the csproj as metadata, but all *.cs files instead. Which is a very limited solution.

Is there an idea of how a non-workaround solution would look like?

[alanedwardes]

@rschili Is it an option for you to point docfx at your compiled binaries? To avoid touching any environment variables as above I settled with that solution - as part of the build, I get docfx to generate metadata for the debug binaries in bin. A bit more concise than looking at source files.

[OskarKlintrot]

Does DocFX have to use MSBuild for any other reason than building the projects? If not, why can't you look in the .csproj if the project is a dotnet core project and if so use the dotnet tooling instead?

[NZSmartie]

Based on your guys comments and a bit of digging around. I created a Power Shell script to get Visual Studio's install path, use a specified .Net core SDK, and then set up environment variables before invoking docfx for a .Net Core solution.

powershell .\docfx.ps1 docfx.json --serve

# docfx.ps1
$docfxVersion = "2.24.0"
$VisualStudioVersion = "15.0";
$DotnetSDKVersion = "2.0.0";

# Get dotnet paths
$MSBuildExtensionsPath = "C:\Program Files\dotnet\sdk\" + $DotnetSDKVersion;
$MSBuildSDKsPath = $MSBuildExtensionsPath + "\SDKs";

# Get Visual Studio install path
$VSINSTALLDIR =  $(Get-ItemProperty "Registry::HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\VisualStudio\SxS\VS7").$VisualStudioVersion;

# Add Visual Studio environment variables
$env:VisualStudioVersion = $VisualStudioVersion;
$env:VSINSTALLDIR = $VSINSTALLDIR;

# Add dotnet environment variables
$env:MSBuildExtensionsPath = $MSBuildExtensionsPath;
$env:MSBuildSDKsPath = $MSBuildSDKsPath;

# Build our docs
& .\docfx.console.$docfxVersion\tools\docfx @args

It's a little hacky, but it works.

Edit: This has been tested on AppVeyor and works really well. Here is my build script that's invoked from appveyor.yml:

build_script:
 - ps: ./docs/build.ps1 -Deploy

https://github.com/NZSmartie/CoAP.Net/blob/docs/docs/build.ps1

[OskarKlintrot]

Isn't there any other way around this than using VS? We are using Linux on our build machine so we can't exactly install VS...

[vicancy]

@OskarKlintrot #1752 (comment) is a work around without VS, however this is for Windows, I am not sure if the config is also applicable for Linux..

[OskarKlintrot]

I couldn't get it working on W10 when I tried so I doubt it will work on Linux... Thanks anyway!

[rschili]

@alanedwardes sorry it took me so long to give this a shot. This actually works! Great solution, thank you. I didn't realize docfx could feed from dlls directly. And when I heard it can, I was afraid it may be slow like sandcastle was. But it works great and fast.

[daveaglick]

I'm seeing this exact same problem when using the MSBuild API, totally unrelated to DocFx:

C:\Program Files\dotnet\sdk\2.0.2\Sdks\Microsoft.NET.Sdk\Sdk\Sdk.props(29,3): error MSB4019: The imported project "C:\Program Files\dotnet\sdk\2.0.2\2.0\Microsoft.Common.props" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk.

And like @jskeet it's not happening locally, only on AppVeyor (which does have Visual Studio Community 2017 15.4 installed). It's also only happening for some projects but not others. It's like for some projects, the MSBuildToolsVersion MSBuild property gets set to "2.0" instead of "15.0".

So in summary, this doesn't look specific to DocFx but rather a problem with the MSBuild API. However, I can't find anything specifically discussing this over at MSBuild which is why I'm here (though a lot of related issues). Does anyone have any idea why this is happening?

[mellinoe]

Could we get a summary of the appropriate hack to be using here? I've followed a couple of tips above, but they aren't actually working (as others have already replied).

Unless I'm missing something, this is a pretty fundamental scenario that isn't working. I'm not trying to do anything unusual, I just have a bog standard netstandard2.0 library that I added to a blank "docfx init" project...

[kdahltx]

I don't like the solution I got to work but in projects created with VS2017 I noticed it no longer lists all of the files in the project. It just picks up the .cs files. I added and list the files with the <Compile Include="filexxx.cs /> entries for each file. It now works for that project and only gets the files I added to the .csproj file.

[aravind-rnd]

error MSB4019: The imported project "/common.props" was not found. Confirm that the expression in the Import declaration "../../common.props" is correct, and that the file exists on disk.