[mlpack] issue with mlpack Python bindings on Mac OsX

Frodo Jedi frodojedi.mailinglist at gmail.com
Sat Dec 15 10:13:58 EST 2018 

Hi Ryan,
running make python with sudo gives no errors.

Surely I was building as the same user that I configured the project with.
I could not understand what you were referring to with crsutils and
especially what links on overflow you were referring to. I am not sure what
should I search for.

To avoid permission issues for the file I tried to run the whole building
procedure with sudo. Here the output:

sudo cmake ../ gives the following:

-- The C compiler identification is AppleClang 9.0.0.9000039
-- The CXX compiler identification is AppleClang 9.0.0.9000039
-- Check for working C compiler:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Check for working C compiler:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- Check for working CXX compiler:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- cotire 1.7.10 loaded.
-- Found Armadillo: /usr/local/lib/libarmadillo.dylib (found suitable
version "9.100.5", minimum required is "6.500.0")
-- Armadillo libraries: /usr/local/lib/libarmadillo.dylib
-- Boost version: 1.68.0
-- Found the following Boost libraries:
--   program_options
--   unit_test_framework
--   serialization
-- Could NOT find OpenMP_C (missing: OpenMP_C_FLAGS OpenMP_C_LIB_NAMES)
-- Could NOT find OpenMP_CXX (missing: OpenMP_CXX_FLAGS
OpenMP_CXX_LIB_NAMES)
-- Could NOT find OpenMP (missing: OpenMP_C_FOUND OpenMP_CXX_FOUND)
-- Found Git: /usr/local/bin/git (found version "2.19.0")
CMake Warning at CMakeLists.txt:425 (message):
  txt2man not found; man pages will not be generated.


-- Found PythonInterp: /usr/local/bin/python (found version "2.7.15")
-- Found Python: /usr/local/bin/python
-- Found PY_distutils: /usr/local/Cellar/python at 2
/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils
-- Found PY_Cython: /usr/local/lib/python2.7/site-packages/Cython
-- Found PY_numpy: /usr/local/lib/python2.7/site-packages/numpy
-- Found PY_pandas: /usr/local/lib/python2.7/site-packages/pandas
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY - Success
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY - Success
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Success
-- CXX target mlpack cotired.
-- CXX target mlpack_test cotired.
-- Found Doxygen: /usr/local/bin/doxygen (found version "1.8.14") found
components:  doxygen missing components:  dot
-- Found PkgConfig: /usr/local/bin/pkg-config (found version "0.29.2")
-- Configuring done
-- Generating done
-- Build files have been written to:
/Users/frodo/Documents/Machine_learning/mlpack-3.0.4/build





sudo make install gives the following errors:


running install
Checking .pth file support in usr/local/lib/python2.7/site-packages/
/usr/local/opt/python at 2/bin/python2.7 -E -c pass
TEST FAILED: usr/local/lib/python2.7/site-packages/ does NOT support .pth
files
error: bad install directory or PYTHONPATH

You are attempting to install a package to a directory that is not
on PYTHONPATH and which Python does not read ".pth" files from.  The
installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

    usr/local/lib/python2.7/site-packages/

and your PYTHONPATH environment variable currently contains:

    '/usr/local/lib/python2.7/site-packages/'

Here are some of your options for correcting the problem:

* You can choose a different installation directory, i.e., one that is
  on PYTHONPATH or supports .pth files

* You can add the installation directory to the PYTHONPATH environment
  variable.  (It must then also be on PYTHONPATH whenever you run
  Python and want to use the package(s) you are installing.)

* You can set up the installation directory to support ".pth" files by
  using one of the approaches described here:


https://setuptools.readthedocs.io/en/latest/easy_install.html#custom-installation-locations


Please make the appropriate changes for your system and try again.
-- Installing: /usr/local/bin/mlpack_adaboost
-- Installing: /usr/local/bin/mlpack_approx_kfn
-- Installing: /usr/local/bin/mlpack_cf
-- Installing: /usr/local/bin/mlpack_dbscan
-- Installing: /usr/local/bin/mlpack_decision_stump
-- Installing: /usr/local/bin/mlpack_decision_tree
-- Installing: /usr/local/bin/mlpack_det
-- Installing: /usr/local/bin/mlpack_emst
-- Installing: /usr/local/bin/mlpack_fastmks
-- Installing: /usr/local/bin/mlpack_gmm_train
-- Installing: /usr/local/bin/mlpack_gmm_generate
-- Installing: /usr/local/bin/mlpack_gmm_probability
-- Installing: /usr/local/bin/mlpack_hmm_train
-- Installing: /usr/local/bin/mlpack_hmm_loglik
-- Installing: /usr/local/bin/mlpack_hmm_viterbi
-- Installing: /usr/local/bin/mlpack_hmm_generate
-- Installing: /usr/local/bin/mlpack_hoeffding_tree
-- Installing: /usr/local/bin/mlpack_kernel_pca
-- Installing: /usr/local/bin/mlpack_kmeans
-- Installing: /usr/local/bin/mlpack_lars
-- Installing: /usr/local/bin/mlpack_linear_regression
-- Installing: /usr/local/bin/mlpack_local_coordinate_coding
-- Installing: /usr/local/bin/mlpack_logistic_regression
-- Installing: /usr/local/bin/mlpack_lsh
-- Installing: /usr/local/bin/mlpack_mean_shift
-- Installing: /usr/local/bin/mlpack_nbc
-- Installing: /usr/local/bin/mlpack_nca
-- Installing: /usr/local/bin/mlpack_knn
-- Installing: /usr/local/bin/mlpack_kfn
-- Installing: /usr/local/bin/mlpack_nmf
-- Installing: /usr/local/bin/mlpack_pca
-- Installing: /usr/local/bin/mlpack_perceptron
-- Installing: /usr/local/bin/mlpack_preprocess_split
-- Installing: /usr/local/bin/mlpack_preprocess_binarize
-- Installing: /usr/local/bin/mlpack_preprocess_describe
-- Installing: /usr/local/bin/mlpack_preprocess_imputer
-- Installing: /usr/local/bin/mlpack_radical
-- Installing: /usr/local/bin/mlpack_random_forest
-- Installing: /usr/local/bin/mlpack_range_search
-- Installing: /usr/local/bin/mlpack_krann
-- Installing: /usr/local/bin/mlpack_softmax_regression
-- Installing: /usr/local/bin/mlpack_sparse_coding




I can't figure out what exactly I have to do. The variable PYTHONPATH is
empty on my system, not sure what is the best way/how to set it.

Also, it is not clear why do I have to build with sudo in order to avoid
the permission issue.

Any help/recommendation?





On Fri, Dec 14, 2018 at 5:34 AM Ryan Curtin <ryan at ratml.org> wrote:

> On Thu, Dec 13, 2018 at 10:26:24AM +0000, Frodo Jedi wrote:
> > Hi Ryan,
> > many thanks for your quick reply.
> >
> > When running "make python"  I get the following error:
> >
> > clang -fno-strict-aliasing -fno-common -dynamic -g -O2 -DNDEBUG -g
> -fwrapv
> > -O3 -Wall -Wstrict-prototypes
> > -I/usr/local/lib/python2.7/site-packages/numpy/core/include
> >
> -I/Users/frodo/Documents/Machine_learning/mlpack-3.0.4/build/src/mlpack/bindings/python/
> > -I/Users/frodo/Documents/Machine_learning/mlpack-3.0.4/src
> > -I/usr/local/include -I/usr/local/include
> >
> -I/Users/frodo/Documents/Machine_learning/mlpack-3.0.4/build/src/mlpack/..
> > -I/usr/local/include -I/usr/local/opt/openssl/include
> > -I/usr/local/opt/sqlite/include
> > -I/usr/local/Cellar/python at 2
> /2.7.15_1/Frameworks/Python.framework/Versions/2.7/include/python2.7
> > -c
> >
> /Users/frodo/Documents/Machine_learning/mlpack-3.0.4/build/src/mlpack/bindings/python/mlpack/arma_numpy.cpp
> > -o
> >
> build/temp.macosx-10.12-x86_64-2.7/Users/frodo/Documents/Machine_learning/mlpack-3.0.4/build/src/mlpack/bindings/python/mlpack/arma_numpy.o
> > -DBINDING_TYPE=BINDING_TYPE_PYX -std=c++11 -O3 -Wno-unknown-pragmas
> > error: unable to open output file
> >
> >
> 'build/temp.macosx-10.12-x86_64-2.7/Users/frodo/Documents/Machine_learning/mlpack-3.0.4/build/src/mlpack/bindings/python/mlpack/arma_numpy.o':
> >       'Operation not permitted'
> > 1 error generated.
> > error: command 'clang' failed with exit status 1
>
> So, this is the actual error here.  Don't worry about the numpy
> warnings, they are expected.  I think this is actually a result of
> Cython continuing to use deprecated numpy functionality.  I am not sure
> when (or if) that will be fixed.
>
> In any case, it's rather strange for clang to fail claiming 'operation
> not permitted' on simply building a .o file.  It's also written "error:
> unable to open output file", so is it possible that there is a
> permissions issue?  Are you building as the same user that you
> configured the project with?
>
> Some brief StackOverflow searches suggest that this could have something
> to do with 'csrutil', but I am not an OS X expert so make sure you do
> some reading before attempting anything there.  Another thing to try
> might be to try and create the file 'arma_numpy.o' (just as an empty
> file or whatever, with the 'touch' command) in the directory that CMake
> is trying to build it in.  That might give some more information about
> the problem, and you might be able to diagnose and fix it from there.
>
> > Notably, there are some issues when running cmake whit a completely clean
> > (i.e. new) build directory:
> >
> > -- Could NOT find OpenMP_C (missing: OpenMP_C_FLAGS OpenMP_C_LIB_NAMES)
> > -- Could NOT find OpenMP_CXX (missing: OpenMP_CXX_FLAGS
> > OpenMP_CXX_LIB_NAMES)
> > -- Could NOT find OpenMP (missing: OpenMP_C_FOUND OpenMP_CXX_FOUND)
>
> Don't worry about those, it means your compiler does not seem to have
> adequate OpenMP support.  That's ok, and mlpack will still compile just
> fine.  If it was an error, it would have stopped configuration and you
> would not be able to compile.
>
> > What do you suggest should I do? Any flag for cmake you reccommend me to
> > use?
>
> Everything else in the output looks okay to me, so no recommendations
> for changed CMake flags or anything.  The actual build failure you were
> seeing is strange to me and suggests some issue on the system itself (at
> least that it was it looks like to me).
>
> In any case, I'm happy to see that CMake has correctly found the right
> Python packages during the configuration step.
>
> Hope that this helps!
>
> Thanks,
>
> Ryan
>
> --
> Ryan Curtin    | "So I got that going for me, which is nice."
> ryan at ratml.org |   - Carl Spackler
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://knife.lugatgt.org/pipermail/mlpack/attachments/20181215/ff6c148d/attachment.html>

More information about the mlpack mailing list