#313 enabled fix differing versions of dependency scikit learn

[ckunki]

Closes #313

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[ckunki]

During nbtests in CI we observed the error message

E    ValueError: The feature names should match those that were passed during fit.
E    Feature names unseen at fit time:
E    - 10
E    - 11
E    - 2
E    - 3
E    - 4
E    - ...
E    Feature names seen at fit time, yet now missing:
E    - FALPHA
E    - FASYM
E    - FCONC
E    - FCONC1
E    - FDIST
E    - ...

[ahsimb]

The specific version of the library used by the default SLC depends ...

"AI-Lab adapts its own version of the library if required." I don't think it's a good explanation of what is happening here. AI-Lab itself is not doing anything. The user should run the script like the one in this notebook to make sure the versions are aligned.

Reply via ReviewNB

[ckunki]

I updated the paragraph in the next push.
Please review again.

[tkilias]

@ahsimb thx, that was also what I meant needs to be formulated differently and @ckunki for changing

[ahsimb]

Can you split this code cell in two parts and add markdown cells for each part explaining the code?

In the first part we create a UDF that reads the version of the scikit_learn library. The UDF runs inside the language container, therefore the library version detected by the UDF is the version installed in this container.

In the second part we compare the version returned by the UDF with the version in the AI-Lab environment. If they differ we install the UDF's version in the AI-Lab environment.

BTW, do we need to run pip with the --upgrade option?

Reply via ReviewNB

[ckunki]

Sure, see next push.

[ckunki]

BTW, do we need to run pip with the --upgrade option?

That I cannot answer. My tests have been successful without this option.

[ahsimb]

Looks good to me

[ahsimb]

Line #12.    model.fit(X_train.values, y_train)

Why? The model is happy to take a DataFrame as an input.

Reply via ReviewNB

[ckunki]

We observed an error message as specified in comment-2293085752.
The notebook sklearn_train_abalone.ipynb did use .values already.

[ahsimb]

Maybe earlier versions of scikit-learn didn't care about feature names. It seems that when we read input in udf like ctx.get_dataframe(num_rows=1000, start_col=1) we do not get the column names in the pandas DataFrame. That's strange.