Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dpctl pip install broken on windows for python3.11 #1745

Closed
icfaust opened this issue Jul 19, 2024 · 5 comments
Closed

dpctl pip install broken on windows for python3.11 #1745

icfaust opened this issue Jul 19, 2024 · 5 comments

Comments

@icfaust
Copy link

icfaust commented Jul 19, 2024

System info:

OS Name:                   Microsoft Windows 11 Enterprise
OS Version:                10.0.22631 N/A Build 22631
System Type:               x64-based PC
Processor(s):              1 Processor(s) Installed.
                           [01]: Intel64 Family 6 Model 140 Stepping 1 GenuineIntel ~2995 Mhz

Using python3.11

Reproducer

python -m venv env2
env2\Scripts\activate.bat
python -m pip install dpctl
python -c "import dpctl"

image

Not sure if I am doing something wrong, but I am seeing this failure locally and in github action VMs using pip

@icfaust
Copy link
Author

icfaust commented Jul 19, 2024

temporary fix:

import os, site
path_to_env = site.getsitepackages()[0]
path_to_libs = os.path.join(path_to_env, "Library", "bin")
os.add_dll_directory(path_to_libs)
import dpctl

@icfaust
Copy link
Author

icfaust commented Jul 21, 2024

here is a rough github workflow which can show the error. also when fixed it doesnt show any devices (second error? i was able to install manually the dpcpp compiler separately using the offline installer and yield an opencl device via sycl-ls). also broken for 3.9

name: pip dpctl validator
on:
  workflow_dispatch:

env:
  PYTHON_VERSION: 3.9
  DPCTL_VERSION: 0.17.0

jobs:
  dpctl:
    name: dpctl_test
    runs-on: windows-2022
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Install Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ env.PYTHON_VERSION }}
      - name: Install dpctl
        shell: cmd
        run: |
          set PATH=C:\msys64\usr\bin;%PATH%
          python -m venv env2
          call .\env2\Scripts\activate.bat
          set DPCTL_SUPPORTED=3.9 3.11
          FOR %%i IN (%DPCTL_SUPPORTED%) DO if ${{ env.PYTHON_VERSION }}==%%i pip install dpctl==${{ env.DPCTL_VERSION }}
          python -c "import os, os.path, site; path_to_env = site.getsitepackages()[0]; path_to_libs = os.path.join(path_to_env, 'Library', 'bin'); temp=os.add_dll_directory(path_to_libs) if os.path.exists(path_to_libs) else None; import dpctl; print(dpctl.get_devices())"
          python -c "import dpctl; print(dpctl.get_devices())"
          pip list
      - name: Install dpctl conda
        shell: cmd
        run: |
          set PATH=C:\msys64\usr\bin;%PATH%
          call C:\Miniconda\condabin\conda.bat config --add channels https://software.repos.intel.com/python/conda/
          call C:\Miniconda\condabin\conda.bat config --append channels conda-forge https://software.repos.intel.com/python/conda/
          call C:\Miniconda\condabin\conda.bat create --name dpctl_env dpctl
          call C:\Miniconda\condabin\conda.bat activate dpctl_env
          python -c "import dpctl; print(dpctl.lsplatform(2))"

@icfaust
Copy link
Author

icfaust commented Jul 21, 2024

it looks like conda on windows has the same behavior

EDIT: erroneous claim, this issue is addressed in another Turns out that there is a bug in the intel channel for windows and the vc14_runtime

@icfaust
Copy link
Author

icfaust commented Jul 22, 2024

Second issue with missing devices discussed here: #1751

oleksandr-pavlyk added a commit that referenced this issue Dec 4, 2024
Since location of Library\bin in virtual environment is not on
the default search path, importing of dpctl failes due to unmet
dependencies for native extensions of dpctl submodules.

This change introduces _init_helper.py which implements the following
logic using built-in os Python module:

1. If os.add_dll_directory exists, and VIRTUAL_ENV environment variable
   is set, and os.path.join(os.environ["VIRTUAL_ENV"], "Library", "bin")
   exists, call os.add_dll_directory with that directory.

With this change the gh-1745 is fixed, and "python -m dpctl -f" works
out of the box.

Only GPU devices are visible, and to enable CPU device two additional
steps must be performed:

  1. Edit %VIRUAL_ENV%\Library\bin\cl.cfg and set CL_CONFIG_TBB_DLL_PATH
     variable at the bottom of the configuration file to the expanded
     value of %VIRUAL_ENV%\Library\bin\tbb12.dll but use forward slashes,
     instead of native backward slashes.
  2. Append %VIRUAL_ENV%\Library\bin to the PATH using
     `set "PATH=%PATH%:%VIRTUAL_ENV%\Library\bin"`

After these changes `python -m dpctl -f` should see CPU device.
oleksandr-pavlyk added a commit that referenced this issue Dec 7, 2024
Since location of Library\bin in virtual environment is not on
the default search path, importing of dpctl failes due to unmet
dependencies for native extensions of dpctl submodules.

This change introduces _init_helper.py which implements the following
logic using built-in os Python module:

1. If os.add_dll_directory exists, and VIRTUAL_ENV environment variable
   is set, and os.path.join(os.environ["VIRTUAL_ENV"], "Library", "bin")
   exists, call os.add_dll_directory with that directory.

With this change the gh-1745 is fixed, and "python -m dpctl -f" works
out of the box.

Only GPU devices are visible, and to enable CPU device two additional
steps must be performed:

  1. Edit %VIRUAL_ENV%\Library\bin\cl.cfg and set CL_CONFIG_TBB_DLL_PATH
     variable at the bottom of the configuration file to the expanded
     value of %VIRUAL_ENV%\Library\bin\tbb12.dll but use forward slashes,
     instead of native backward slashes.
  2. Append %VIRUAL_ENV%\Library\bin to the PATH using
     `set "PATH=%PATH%:%VIRTUAL_ENV%\Library\bin"`

After these changes `python -m dpctl -f` should see CPU device.
@oleksandr-pavlyk oleksandr-pavlyk mentioned this issue Dec 7, 2024
8 tasks
@oleksandr-pavlyk
Copy link
Collaborator

Closing as completed via gh-1922.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants