[WIP] docs/apple-silicon.md: Add doc on loading CIEL on Apple Silicon #52

dotinspace · 2024-07-31T14:22:24Z

The document is not complete, but the process for loading CIEL into SBCL REPL has been verified to work. It's a start.

I did not manange to make building the binary and/or core image work, yet.

The document is not complete, but the process for loading CIEL into SBCL REPL has been verified to work. I did not manange to make building the binary and/or core image work, yet.

bo-tato · 2024-07-31T15:10:57Z

a simple hack to build it on mac if you are getting an error from a dependency of nodgui, but don't plan on actually using nodgui, is just to comment out the :nodgui line in :depends-on section of file ceil.asd

dotinspace · 2024-07-31T17:11:47Z

Thanks for the suggestion, but it does appear that the GLUT error mentioned in the doc is just the beginning of a longer chain of issues. Removing :nodgui from :depends-on in ciel.asd did not work, but generated other errors, such as: #<LIBRARY CORESERVICES> does not have a known shared library file path, which seems odd.

vindarel · 2024-08-26T15:06:11Z

Hi, thanks for this!

It looks like we should not include nodgui, to ease installation for many people, right?

I envisioned a set of little tools and ready-to-use scripts built with a lil' nodgui GUI, but… that isn't happening, to be honest. I use… one, but actually not so much.

If those tools arise, we'll study a way to build CIEL for the required platformes in a second time.

vindarel · 2024-08-26T15:06:28Z

docs/apple-silicon.md

+$ brew update && brew install \
+       zstd \
+       gcc \
+       sdl2 \


I'm surprised. Why sdl2?

To be honest, I am not sure, I can try without it again, but need some time - incapacitated at the moment. The reason I added it, was I encountered some errors related to SDL, but it might have been dated install via homebrew or me being sort of new to the CL ecosystem, residue from brute force testing. . I will try without it. :)

sdl2 is listed in the dependencies for nodgui? https://www.autistici.org/interzona/nodgui.html

sdl2 is listed in the dependencies for nodgui? https://www.autistici.org/interzona/nodgui.html

indeed! thanks.

dotinspace · 2024-08-26T15:33:33Z

Hi, thanks for this!

It looks like we should not include nodgui, to ease installation for many people, right?

I am not really in any position to judge, but if it is not consistent across platforms, it might be a good idea to hold off for now, and revisit as the need arises.

I envisioned a set of little tools and ready-to-use scripts built with a lil' nodgui GUI, but… that isn't happening, to be honest. I use… one, but actually not so much.

If those tools arise, we'll study a way to build CIEL for the required platformes in a second time.

Sounds like a good idea! My incentive to use CIEL is more terminal centric (and also mainly on BSD), so I don't really need a GUI lib, but it would be swell to have in the future, indeed. Also, bear in mind, it might very well be the setup on the Mac I used that is troublesome, a bit hard to tell, unless there are others experiencing the same.

drops SDL, SDL TTF and JPEG turbo libraries. the flip side of the coin will be the loss of canvas with fast pixel drawing, hardware accelerated canvas for 3D rendering and also the support for JPEG bitmap. for #56 for #52

ccqpein

Add some comment depend on my personal experience.

ccqpein · 2024-09-14T18:47:23Z

docs/apple-silicon.md

+### ASDF
+```example
+$ mkdir -p ~/common-lisp && cd ~/common-lisp
+$ git clone https://gitlab.common-lisp.net/asdf/asdf.git


We need to install the latest asdf to sbcl. Because the sbcl which installed through homebrew has the asdf version 3.3.1 (I don't know why sbcl don't update it for years). So the make ql-deps will return the error that the asdf version is too old (it needs at least 3.3.4).

I reinstall asdf to sbcl every time the sbcl updated on homebrew.

cd ~/common-lisp/asdf make ./tools/asdf-tools install-asdf sbcl

then the sbcl --script check-asdf-version.lisp in ciel folder will return the newest version (3.3.7.1)

Hmm... I did not encounter that problem. The only thing I did with the homebrew sbcl install is to make sure quicklisp is installed and initialised via .sbclrc, then clone asdf into ~/common-lisp, or whatever other location one has chosen to use. I must be missing something, because I end up with (asdf:asdf-version) ; 3.3.7.1. Am I going about it the wrong way?

I reinstalled SBCL through Homebrew, and the ASDF version was reverted to 3.3.1. I had studied the issue before, I believe the problem lies in this code, which includes the contrib/ folder (specifically located at /opt/homebrew/Cellar/sbcl/2.4.8/lib/sbcl/sbclrc). The SBCL version of ASDF has been stuck at version 3.3.1 for years.

I maybe miss something. I would love to sbcl use newest asdf automatically rather than update every time by myself.

I see! You make a fair point. However, it seems there's been multiple discussions about SBCL and ASDF, e.g.: https://bugs.launchpad.net/sbcl/+bug/1988892. Also:

You can update your own ASDF at any time, if your .asd depend on something new, insert a version assertion in them.

The goal of SBCL is to provide a CL implementation, and supplying third party software with it is out of such scope. The current ASDF works fine for SBCL's own needs, loading and building contribs, anything else is on the third party. Would've been nice if ASDF behaved differently, but it doesn't, so that's the current state. And I'm not the bigger person to accept that, so there's also that, but why should I?

Not sure what would solve the problem for you, to be honest. Apart from maybe adding an issue/pr to the Homebrew formula. Anyway, we can add a note about this in the doc?

Yea.. I think I read this issue in sbcl maillist too. I am thinking about give some contributions to homebrew. But I know nothing about ruby 🙃.

themarcelor · 2024-12-23T18:14:22Z

Any updates on this PR?

I just got a new M4 laptop and I can't build the docker image (can't generate a new binary) anymore 😢
(halp)

dotinspace · 2024-12-23T18:50:55Z

@themarcelor Any chance of sharing some specific errors you are encountering?

themarcelor · 2024-12-23T19:22:52Z

Hi @dotinspace ,

The error is kinda obscure. Do you have any pointers?

58.07 ; Loading "magic-ed"
58.07 [package magic-ed]
58.08 To load "ciel/repl":
58.08   Load 1 ASDF system:
58.08     ciel/repl
58.08 ; Loading "ciel/repl"
58.10 ......
58.17 Registering built-in scripts in src/scripts/ …
58.17   scripts: registering apipointer
58.17   scripts: registering finder
58.17   scripts: registering quicksearch
58.17   scripts: registering simpleHTTPserver
58.17   scripts: registering webapp-notify
58.17   scripts: registering webapp
58.18 .
58.47 [undoing binding stack and other enclosing state... mmap: wanted 89128960 bytes at 0x1000000000, actually mapped at 0xffffa8000000
58.56 Dump of /proc/self/maps:
58.56 00400000-00440000 r-xp 00000000 00:3e 111335                             /usr/local/bin/sbcl
...
58.56 121d9000-121fa000 rw-p 00000000 00:00 0                                  [heap]
58.56 f0000000-f0010000 rwxp 00010000 00:3e 111403                             /usr/local/lib/sbcl/sbcl.core
<hundreds of lines with the same nonsense>
58.56 ffffad600000-ffffad86b000 r-xp 00000000 00:3e 21620                      /usr/lib/aarch64-linux-gnu/libcrypto.so.1.1
...
58.56 ffffadf3a000-ffffadfba000 r-xp 00000000 00:3e 21806                      /usr/lib/aarch64-linux-gnu/libssl.so.1.1
...
58.56 ffffae4d2000-ffffae4e5000 r-xp 00000000 00:3e 120521                     /lib/aarch64-linux-gnu/libresolv-2.31.so
....
58.56 ffffae4f9000-ffffae4fe000 r-xp 00000000 00:3e 120492                     /lib/aarch64-linux-gnu/libnss_dns-2.31.so
...
...58.56 ffffae701000-ffffae703000 rw-p 0015f000 00:3e 120455                     /lib/aarch64-linux-gnu/libc-2.31.so
...
58.56 ffffae7b0000-ffffae7b1000 rw-p 0009a000 00:3e 120467                     /lib/aarch64-linux-gnu/libm-2.31.so
...
58.56 ffffae7da000-ffffae7db000 rw-p 00019000 00:3e 14077                      /lib/aarch64-linux-gnu/libz.so.1.2.11
58.56 ffffae7db000-ffffae7f7000 r-xp 00000000 00:3e 120515                     /lib/aarch64-linux-gnu/libpthread-2.31.so
...
58.56 ffffae80c000-ffffae80f000 r-xp 00000000 00:3e 120461                     /lib/aarch64-linux-gnu/libdl-2.31.so
...
58.56 ffffae84e000-ffffae850000 r--p 00000000 00:00 0                          [vvar]
58.56 ffffae850000-ffffae851000 r-xp 00000000 00:00 0                          [vdso]
58.56 ffffae851000-ffffae852000 r--p 00021000 00:3e 120430                     /lib/aarch64-linux-gnu/ld-2.31.so
58.56 ffffae852000-ffffae854000 rw-p 00022000 00:3e 120430                     /lib/aarch64-linux-gnu/ld-2.31.so
58.56 ffffd36bf000-ffffd36e0000 rw-p 00000000 00:00 0                          [stack]
58.56 fatal error encountered in SBCL pid 180 tid 180:
58.56 remap_free_pages: page moved, 0x1000000000 ==> (nil)
58.56
58.56 done]
58.56    0:     0xffffae001710 pc=0x10019750e0
...
58.56   26:     0xffffae000000 pc=0x1000053ff0
58.57 make: *** [Makefile:90: build] Error 1
------
Dockerfile:16
--------------------
  15 |
  16 | >>> RUN make ql-deps \
  17 | >>>     && make build \
  18 | >>>     && cp bin/* /usr/local/bin/
  19 |
--------------------
ERROR: failed to solve: process "/bin/sh -c make ql-deps     && make build     && cp bin/* /usr/local/bin/" did not complete successfully: exit code: 2

themarcelor · 2024-12-23T19:39:20Z

@dotinspace If I comment out the build lines from the Dockerfile:

RUN make ql-deps
#    && make build \
#    && cp bin/* /usr/local/bin/

Then I can build the image (docker build -t ciel .) and run it (docker run -it ciel /bin/bash) to try the build inside the container:

CIEL [master●] docker run -it ciel /bin/bash
root@502a74c05281:/home# make build

But I get the same result. At some point I added the --dynamic-space-size 2024 instruction to the sbcl command defined in the Makefile and the make build step completed without errors but then the binary is just an empty / zero byte file 😕 . If I delete the empty ciel file, then I can reproduce the error again with make build:

fatal error encountered in SBCL pid 162 tid 162:
remap_free_pages: page moved, 0x1000000000 ==> (nil)

   0:     0xffffaf001710 pc=0x10019750e0
   1:     0xffffaf0015e8 pc=0x1001467970
   2:     0xffffaf0015a0 pc=0x1008e9e160

Looks like some new lib or a new gcc version that is pulled by the apt-get install does not play well with this version of SBCL we are using as the parent image 🤔 😢
https://bugs.launchpad.net/sbcl/+bug/2064524

But, clfoundation/sbcl does not give us any other option:
https://hub.docker.com/r/clfoundation/sbcl/tags

Unless we pull the parent image from some other repo. @vindarel what do you think?

themarcelor · 2024-12-23T23:03:53Z

Ok, I finally managed to build the binary but I had to fiddle with a few files....
Although I had to use some other Docker image with a newer version of SBCL -> fukamachi/sbcl:2.3.8.
(At first I tried 2.3.7 but that had a bug associated with vgplot).

Here's a branch that will allow Mac OS + Silicon users build the CIEL binary 💪🏼
master...themarcelor:CIEL:fix-docker-build-for-apple-silicon

This is still awkward 😅

QLDIR ?= ~/.roswell/lisp/quicklisp/local-projects

I gotta figure out how to reconcile the quicklisp/local-projects auto-scanning with the existing ~/quicklisp references.
But, for now, I'm no longer blocked and I can finally build new binaries with my new M4 laptop 😌 🚀

dotinspace · 2024-12-23T23:24:33Z

Glad you got it sorted. The error message was indeed a bit cryptic, sorry I couldn't be of any asssistance.

themarcelor · 2024-12-23T23:36:53Z

Np @dotinspace , I appreciate the soundboarding / handholding 🤝

So, I guess we could tweak this PR if @vindarel agrees with the adoption of a new SBCL version + New parent image for our Docker build. Let me know how you guys want to proceed 👍🏼

dotinspace · 2024-12-25T12:08:12Z

Would be good to add steps/instruction on how to do it that way in the doc, indeed!

themarcelor · 2024-12-27T16:33:09Z

Hi @dotinspace , I made some adjustments to my branch:
master...themarcelor:CIEL:fix-docker-build-for-apple-silicon

Feel free to copy over the instructions to your PR if you find this suitable.

Here is a breakdown of all the changes:

In the Dockerfile, we are now adopting a new parent image (FROM fukamachi/sbcl:2.3.8 AS build) so we can upgrade the SBCL version.

I also had to add libreadline-dev to the list of packages installed by apt-get.

This new fukamachi/sbcl parent image does not have the /usr/local/bin/install-quicklisp script baked in it. It has the Roswell - Common Lisp environment setup Utility pre-installed with its own quicklisp available under the ~/.roswell/lisp/quicklisp path. Hence, I had to add some steps to the Dockerfile in order to adapt to this new setup.

RUN mv src/scripts/sbclrc ~/.sbclrc \
    && mv src/scripts/sbcl_ros_wrapper /usr/local/bin/sbcl \
    && ln -sf ~/.roswell/lisp/quicklisp ~/quicklisp

Basically, I created a new sbclrc file to override the default quicklisp/local-projects path to point to the original path in which we place the CIEL dependencies (QLDIR ?= $(HOME)/quicklisp/local-projects). This is done with the following command: (setf ql:*local-project-directories* '("/root/quicklisp/local-projects"));.

However, this Roswell thing does NOT load the ~/.sbclrc init file when its sbcl is executed, hence, we need to override the ros script to set the new value of the *local-project-directories* variable. So the goal is to replace the /usr/local/bin/sbcl script with:

#!/bin/sh
exec ros +R -l ~/.sbclrc run -- "$@"

And then we just create a symlink to make all CIEL scripts find quicklisp in ~/quicklisp instead of ~/.roswell/lisp/quicklisp. Which is done with ln -sf ~/.roswell/lisp/quicklisp ~/quicklisp.

The final change is to replace cp bin/* /usr/local/bin/ with cp ./ciel /usr/local/bin/ because the ciel binary is no longer found under a $HOME/bin folder.

If we could get a new a new clfoundation/sbcl image with a new sbcl version (sbcl:2.3.8), then this would be significantly easier.

themarcelor · 2024-12-27T17:43:31Z

Like... who do we talk to in order to get a new clfoundation/sbcl image?
I see a PR here:
cl-docker-images/sbcl#3
but it was closed without any comments 😢

According to their CONTRIBUTING.md, looks like they accept PRs in https://gitlab.common-lisp.net/cl-docker-images 🤔

themarcelor · 2024-12-27T17:55:53Z

Ok, I just sent a request to join https://gitlab.common-lisp.net/
So I can create issues and upvote PRs (and potentially bug @daewok with questions).

Looks like they are cooking a PR though:
https://gitlab.common-lisp.net/cl-docker-images/sbcl/-/merge_requests/1

vindarel · 2025-01-14T20:58:22Z

Hey, thanks for all. It is annoying that, while relying on another Docker image, we have to go to all these hops with Roswell… +1 to try another image or collaborate on PRs, but I trust you guys on this.

docs/apple-silicon.md: Add doc on loading CIEL on Apple Silicon

8fb1dab

The document is not complete, but the process for loading CIEL into SBCL REPL has been verified to work. I did not manange to make building the binary and/or core image work, yet.

vindarel reviewed Aug 26, 2024

View reviewed changes

vindarel mentioned this pull request Aug 26, 2024

Evaluating: removal of nodgui (removed, needs testing of nodgui-lite) #56

Open

vindarel force-pushed the master branch from ab90de0 to 13827a4 Compare August 30, 2024 11:06

vindarel changed the title ~~docs/apple-silicon.md: Add doc on loading CIEL on Apple Silicon~~ [WIP] docs/apple-silicon.md: Add doc on loading CIEL on Apple Silicon Sep 3, 2024

vindarel marked this pull request as draft September 3, 2024 09:22

vindarel force-pushed the master branch 2 times, most recently from 4481c23 to cc99227 Compare September 4, 2024 13:11

ccqpein reviewed Sep 14, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] docs/apple-silicon.md: Add doc on loading CIEL on Apple Silicon #52

[WIP] docs/apple-silicon.md: Add doc on loading CIEL on Apple Silicon #52

dotinspace commented Jul 31, 2024

bo-tato commented Jul 31, 2024

dotinspace commented Jul 31, 2024

vindarel commented Aug 26, 2024

vindarel Aug 26, 2024

dotinspace Aug 26, 2024

bo-tato Aug 26, 2024

vindarel Aug 26, 2024

dotinspace commented Aug 26, 2024

ccqpein left a comment

ccqpein Sep 14, 2024

dotinspace Sep 15, 2024

ccqpein Sep 15, 2024

dotinspace Sep 17, 2024 •

edited

Loading

ccqpein Sep 18, 2024

themarcelor commented Dec 23, 2024

dotinspace commented Dec 23, 2024

themarcelor commented Dec 23, 2024

themarcelor commented Dec 23, 2024

themarcelor commented Dec 23, 2024 •

edited

Loading

dotinspace commented Dec 23, 2024

themarcelor commented Dec 23, 2024

dotinspace commented Dec 25, 2024

themarcelor commented Dec 27, 2024

themarcelor commented Dec 27, 2024 •

edited

Loading

themarcelor commented Dec 27, 2024

vindarel commented Jan 14, 2025 •

edited

Loading

[WIP] docs/apple-silicon.md: Add doc on loading CIEL on Apple Silicon #52

Are you sure you want to change the base?

[WIP] docs/apple-silicon.md: Add doc on loading CIEL on Apple Silicon #52

Conversation

dotinspace commented Jul 31, 2024

bo-tato commented Jul 31, 2024

dotinspace commented Jul 31, 2024

vindarel commented Aug 26, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dotinspace commented Aug 26, 2024

ccqpein left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dotinspace Sep 17, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

themarcelor commented Dec 23, 2024

dotinspace commented Dec 23, 2024

themarcelor commented Dec 23, 2024

themarcelor commented Dec 23, 2024

themarcelor commented Dec 23, 2024 • edited Loading

dotinspace commented Dec 23, 2024

themarcelor commented Dec 23, 2024

dotinspace commented Dec 25, 2024

themarcelor commented Dec 27, 2024

themarcelor commented Dec 27, 2024 • edited Loading

themarcelor commented Dec 27, 2024

vindarel commented Jan 14, 2025 • edited Loading

dotinspace Sep 17, 2024 •

edited

Loading

themarcelor commented Dec 23, 2024 •

edited

Loading

themarcelor commented Dec 27, 2024 •

edited

Loading

vindarel commented Jan 14, 2025 •

edited

Loading