Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Migrate ePub generation to go-epub #679

Merged
merged 21 commits into from
Sep 16, 2023

Conversation

Monirzadeh
Copy link
Collaborator

@Monirzadeh Monirzadeh commented Jul 27, 2023

This pull request solves the problem in #638 and #678
Please before merge test that in real device. Based on your report and in my debug system, it should work fine.
we move to go-epub in this pull request that result of that
fix #638
fix #678
fix #688
fix #635

TODO:

  • add go-epub to requirement
  • Support RTL
  • update HTML to use included image
  • fix no reference image file. duplicate file included
  • update unit test

@Monirzadeh Monirzadeh changed the title fix apple books cannot import generated book with shiori fix #638 fix apple books cannot import generated book with shiori Jul 27, 2023
Copy link
Member

@fmartingr fmartingr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good work @Monirzadeh, just some comments around the identifier.

internal/core/ebook.go Outdated Show resolved Hide resolved
internal/core/ebook.go Outdated Show resolved Hide resolved
@fmartingr
Copy link
Member

I have tested this locally and the resulting epub works in apple books, but I can't see any images (#478) nor in apple books nor in epub-reader.online. Archived version displays images correctly.

Tested URL: https://100r.co/site/weathering_software_winter.html

@Monirzadeh
Copy link
Collaborator Author

I have tested this locally and the resulting epub works in apple books, but I can't see any images (#478) nor in apple books nor in epub-reader.online. Archived version displays images correctly.

Tested URL: https://100r.co/site/weathering_software_winter.html

I test that URL, it is not related to #478.
As I see, apple books don't handle anything itself (for example caliber, show me the image).
I will fix that.

@fmartingr Can you confirm this file work fine in apple books?
book.zip

@fmartingr
Copy link
Member

@fmartingr Can you confirm this file work fine in apple books? book.zip

Imports ok. Pictures not working:

Screenshot 2023-07-30 at 07 25 41

Pictures show ok in epub-reader.online.

Note: Apple Books is not high priority, so don't worry too much about it!

@Monirzadeh Monirzadeh changed the title fix apple books cannot import generated book with shiori Move to go-epub Aug 5, 2023
@fmartingr fmartingr changed the title Move to go-epub refactor: Migrate ePub generation to go-epub Aug 12, 2023
@Monirzadeh
Copy link
Collaborator Author

Monirzadeh commented Sep 16, 2023

Finally :)
sorry if it take so long. please mention anything missing. i test the result to any device i have access that but good if check them twice.

@fmartingr
Copy link
Member

Finally :)
sorry if it take so long. please mention anything missing. i test the result to any device i have access that but good if check them twice.

Shouldn't we make a release of go-epub? 😃

@Monirzadeh
Copy link
Collaborator Author

Finally :)
sorry if it take so long. please mention anything missing. i test the result to any device i have access that but good if check them twice.

Shouldn't we make a release of go-epub? 😃

go-epub v1.20 😃

Copy link
Member

@fmartingr fmartingr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Way less code, this is awesome :D

internal/core/ebook.go Outdated Show resolved Hide resolved
Copy link
Member

@fmartingr fmartingr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome work!

@fmartingr fmartingr merged commit 9e91029 into go-shiori:master Sep 16, 2023
6 checks passed
@Monirzadeh Monirzadeh deleted the fix-apple-books branch September 16, 2023 18:50
truecharts-admin referenced this pull request in truecharts/public Mar 17, 2024
…0@628826c by renovate (#19427)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [ghcr.io/go-shiori/shiori](https://togithub.com/go-shiori/shiori) |
minor | `v1.5.5` -> `v1.6.0` |

---

> [!WARNING]
> Some dependencies could not be looked up. Check the Dependency
Dashboard for more information.

---

### Release Notes

<details>
<summary>go-shiori/shiori (ghcr.io/go-shiori/shiori)</summary>

###
[`v1.6.0`](https://togithub.com/go-shiori/shiori/releases/tag/v1.6.0)

[Compare
Source](https://togithub.com/go-shiori/shiori/compare/v1.5.5...v1.6.0)

It's finally here! After some work we have started moving towards **a
more usable and open API** for others to consume, with **proper session
handling** (no more random logouts on server shutdowns!), improvements
to **ePubs**, UX and some more!

See details below for more information (important notes are the breaking
changes) and please fill an issue if you see anything weird, better be
safe than sorry!

#### Breaking changes

- The `serve` command is considered deprecated and will be removed in a
future release. Right now just proxies to a new `server` command that is
the one that should be used from now on.
- The **server** command uses a new http backend. This **should be
transparent to users** and all things should keep working as usual, but
that meant refactoring some of the underlying systems too, so
experiences may vary between deployments and operating systems. Please
fill an issue if you see that something is not working as expected.
- We are moving the API to a more stable, documented and with a proper
code structure. Right now there's a mix between old and new endpoints
until migration is completed. Check [the
documentation](https://togithub.com/go-shiori/shiori/blob/master/docs/APIv1.md)
for more information on the new API and [this roadmap
filter](https://togithub.com/orgs/go-shiori/projects/2/views/11) to see
progress on the API migration.
- Authentication to the API now uses JWTs instead of session tokens and
the **endpoint has changed to a new one**, please check the
documentation mentioned above. This means that there's no longer logout
issues when the server is restarted or when you log in in other
computer/browser.
- The `--webroot` flag **no longer modifies the routes internally**, is
up to the user to proxy the routes properly to Shiori without the prefix
used to serve it.
That means that if you want to serve Shiori under `domain.com/shiori`
you need to send the path back to Shiori without the `/shiori` prefix so
routes keep working. This can be done in most reverse proxies that we're
aware of. We provided [a sample configuration for
Nginx](https://togithub.com/go-shiori/shiori/blob/master/docs/Configuration.md#reverse-proxies-and-the-webroot-path).
PRs are welcome for other reverse proxies.
- The `shiori/gopher` initial user is a full fledged user instead of
being hardcoded into Shiori. If you want a new user you need to create a
new owner user and then remove the `shiori` user.

#### Release cadence

Right now the release cadence has been slow because we made too many
changes at once and we had to test and finish everything before doing
this release. Plan moving forward is to iterate and release faster so
we're planning smaller milestones to provide new features and fixes
faster to you.

I will post an update announcement when [the
roadmap](https://togithub.com/orgs/go-shiori/projects/2/views/4) is
reviewed.

#### What's Changed

- APIv1: Start working on new REST API. Refactor logic in domains. by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/497](https://togithub.com/go-shiori/shiori/pull/497)
- Run legacy API and new API at the same time. by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/648](https://togithub.com/go-shiori/shiori/pull/648)
- fix: docker buildx tags by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/650](https://togithub.com/go-shiori/shiori/pull/650)
- Allow JWT authentication into legacy APIs by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/651](https://togithub.com/go-shiori/shiori/pull/651)
- Show version in login page by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/652](https://togithub.com/go-shiori/shiori/pull/652)
- fix: package-name in cleanup tag by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/655](https://togithub.com/go-shiori/shiori/pull/655)
- fix: pr tag prune using other action by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/656](https://togithub.com/go-shiori/shiori/pull/656)
- fix: title is never retrieved when adding bookmark by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/664](https://togithub.com/go-shiori/shiori/pull/664)
- Show Shiori version on server command by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/669](https://togithub.com/go-shiori/shiori/pull/669)
- chore: remove irc badge from README by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/674](https://togithub.com/go-shiori/shiori/pull/674)
- fix: title overwritten if user has defined it by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/684](https://togithub.com/go-shiori/shiori/pull/684)
- Proper SQLite default database and warn SHIORI_DBMS users by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/667](https://togithub.com/go-shiori/shiori/pull/667)
- chore: remove verbose logger by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/685](https://togithub.com/go-shiori/shiori/pull/685)
- Fix CI incorrectly tagging RC releases and disables docker builds on
forks by [@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/686](https://togithub.com/go-shiori/shiori/pull/686)
- preserve fragment in URLs
([#&#8203;315](https://togithub.com/go-shiori/shiori/issues/315)) by
[@&#8203;arakimo](https://togithub.com/arakimo) in
[https://github.com/go-shiori/shiori/pull/687](https://togithub.com/go-shiori/shiori/pull/687)
- Swagger improvements by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/666](https://togithub.com/go-shiori/shiori/pull/666)
- fix: Ensure bookmark files are correctly downloaded before deleting
current ones by [@&#8203;Monirzadeh](https://togithub.com/Monirzadeh) in
[https://github.com/go-shiori/shiori/pull/683](https://togithub.com/go-shiori/shiori/pull/683)
- fix(db): handle usage of special characters in searches by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/721](https://togithub.com/go-shiori/shiori/pull/721)
- fix: properly parse mysql connection string, docs update by
[@&#8203;rutkai](https://togithub.com/rutkai) in
[https://github.com/go-shiori/shiori/pull/730](https://togithub.com/go-shiori/shiori/pull/730)
- deps: upgrade to Go 1.21 by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/698](https://togithub.com/go-shiori/shiori/pull/698)
- deps: upgrade github.com/gofrs/uuid to v5 by
[@&#8203;Monirzadeh](https://togithub.com/Monirzadeh) in
[https://github.com/go-shiori/shiori/pull/736](https://togithub.com/go-shiori/shiori/pull/736)
- feat: build css from less files locally by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/735](https://togithub.com/go-shiori/shiori/pull/735)
- refactor: Migrate ePub generation to go-epub by
[@&#8203;Monirzadeh](https://togithub.com/Monirzadeh) in
[https://github.com/go-shiori/shiori/pull/679](https://togithub.com/go-shiori/shiori/pull/679)
- chore(deps): bump the all group with 6 updates by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/go-shiori/shiori/pull/738](https://togithub.com/go-shiori/shiori/pull/738)
- chore(deps): bump the all group with 1 update by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/go-shiori/shiori/pull/740](https://togithub.com/go-shiori/shiori/pull/740)
- feat: use new JWT auth in all frontend API calls by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/743](https://togithub.com/go-shiori/shiori/pull/743)
- chore(deps): bump the all group with 1 update by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/go-shiori/shiori/pull/746](https://togithub.com/go-shiori/shiori/pull/746)
- fix: styles-check and swag-check monitor just needed directory not
project root by [@&#8203;Monirzadeh](https://togithub.com/Monirzadeh) in
[https://github.com/go-shiori/shiori/pull/747](https://togithub.com/go-shiori/shiori/pull/747)
- feat: allow resize the dialogbox for bigger/hidpi screens by
[@&#8203;Monirzadeh](https://togithub.com/Monirzadeh) in
[https://github.com/go-shiori/shiori/pull/732](https://togithub.com/go-shiori/shiori/pull/732)
- feat: allow per-user settings and store them in database by
[@&#8203;Monirzadeh](https://togithub.com/Monirzadeh) in
[https://github.com/go-shiori/shiori/pull/639](https://togithub.com/go-shiori/shiori/pull/639)
- fix: Remove unneeded variable and unify the way send token in header
by [@&#8203;Monirzadeh](https://togithub.com/Monirzadeh) in
[https://github.com/go-shiori/shiori/pull/763](https://togithub.com/go-shiori/shiori/pull/763)
- add create ebook by default in settings by
[@&#8203;Monirzadeh](https://togithub.com/Monirzadeh) in
[https://github.com/go-shiori/shiori/pull/761](https://togithub.com/go-shiori/shiori/pull/761)
- fix: Actions in overlays on mobile hard to press by
[@&#8203;cbe](https://togithub.com/cbe) in
[https://github.com/go-shiori/shiori/pull/759](https://togithub.com/go-shiori/shiori/pull/759)
- fix: Use webp as thumbnail by
[@&#8203;Monirzadeh](https://togithub.com/Monirzadeh) in
[https://github.com/go-shiori/shiori/pull/758](https://togithub.com/go-shiori/shiori/pull/758)
- Update documentation for add links to shiori in android devices from
share menu by [@&#8203;Monirzadeh](https://togithub.com/Monirzadeh) in
[https://github.com/go-shiori/shiori/pull/757](https://togithub.com/go-shiori/shiori/pull/757)
- Fix typos by [@&#8203;shirayu](https://togithub.com/shirayu) in
[https://github.com/go-shiori/shiori/pull/756](https://togithub.com/go-shiori/shiori/pull/756)
- chore(deps): bump the all group with 1 update by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/go-shiori/shiori/pull/767](https://togithub.com/go-shiori/shiori/pull/767)
- refactor: migrate ebook routes by
[@&#8203;Monirzadeh](https://togithub.com/Monirzadeh) in
[https://github.com/go-shiori/shiori/pull/742](https://togithub.com/go-shiori/shiori/pull/742)
- Make suggestions tapable/clickable by
[@&#8203;cbe](https://togithub.com/cbe) in
[https://github.com/go-shiori/shiori/pull/765](https://togithub.com/go-shiori/shiori/pull/765)
- chore: frontend formatting by [@&#8203;cbe](https://togithub.com/cbe)
in
[https://github.com/go-shiori/shiori/pull/764](https://togithub.com/go-shiori/shiori/pull/764)
- ci: add codecov reporting by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/776](https://togithub.com/go-shiori/shiori/pull/776)
- deps: update go dependencies by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/777](https://togithub.com/go-shiori/shiori/pull/777)
- fix typo by [@&#8203;Monirzadeh](https://togithub.com/Monirzadeh) in
[https://github.com/go-shiori/shiori/pull/778](https://togithub.com/go-shiori/shiori/pull/778)
- docs: example deployment for kubernetes by
[@&#8203;JPFrancoia](https://togithub.com/JPFrancoia) in
[https://github.com/go-shiori/shiori/pull/754](https://togithub.com/go-shiori/shiori/pull/754)
- Add Documentation>CLI>Add bookmark by
[@&#8203;LLKoder](https://togithub.com/LLKoder) in
[https://github.com/go-shiori/shiori/pull/794](https://togithub.com/go-shiori/shiori/pull/794)
- fix: generate coverage profile by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/797](https://togithub.com/go-shiori/shiori/pull/797)
- fix: use noreferer to prevent exposing shiori instance url to archived
websites by [@&#8203;istiak101](https://togithub.com/istiak101) in
[https://github.com/go-shiori/shiori/pull/802](https://togithub.com/go-shiori/shiori/pull/802)
- deps: upgrade by [@&#8203;fmartingr](https://togithub.com/fmartingr)
in
[https://github.com/go-shiori/shiori/pull/804](https://togithub.com/go-shiori/shiori/pull/804)
- refactor: migrate bookmark static pages to new http server by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/775](https://togithub.com/go-shiori/shiori/pull/775)
- Fixed lint errors after refactor by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/806](https://togithub.com/go-shiori/shiori/pull/806)
- docs: updated configuration page by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/808](https://togithub.com/go-shiori/shiori/pull/808)
- fix: force usage of shiori prefix for environment variables in
configuration by [@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/807](https://togithub.com/go-shiori/shiori/pull/807)
- deps: updated docker image versions by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/809](https://togithub.com/go-shiori/shiori/pull/809)
- chore(deps): bump the all group with 3 updates by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/go-shiori/shiori/pull/812](https://togithub.com/go-shiori/shiori/pull/812)
- chore(deps): bump the all group with 3 updates by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/go-shiori/shiori/pull/815](https://togithub.com/go-shiori/shiori/pull/815)
- chore(deps): bump the all group with 3 updates by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/go-shiori/shiori/pull/830](https://togithub.com/go-shiori/shiori/pull/830)
- fix: fixes path issues on windows by
[@&#8203;Monirzadeh](https://togithub.com/Monirzadeh) in
[https://github.com/go-shiori/shiori/pull/829](https://togithub.com/go-shiori/shiori/pull/829)
- fix: regressions and documentation from 1.5.5 upgrade by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/837](https://togithub.com/go-shiori/shiori/pull/837)
- fix: update go-epub to latest version to avoid filename errors on
windows by [@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/840](https://togithub.com/go-shiori/shiori/pull/840)
- fix: point go-epub go correct repository by
[@&#8203;fmartingr](https://togithub.com/fmartingr) in
[https://github.com/go-shiori/shiori/pull/842](https://togithub.com/go-shiori/shiori/pull/842)
- feat: allow authentication using proxy request header by
[@&#8203;PterX](https://togithub.com/PterX) in
[https://github.com/go-shiori/shiori/pull/836](https://togithub.com/go-shiori/shiori/pull/836)

#### New Contributors

- [@&#8203;arakimo](https://togithub.com/arakimo) made their first
contribution in
[https://github.com/go-shiori/shiori/pull/687](https://togithub.com/go-shiori/shiori/pull/687)
- [@&#8203;rutkai](https://togithub.com/rutkai) made their first
contribution in
[https://github.com/go-shiori/shiori/pull/730](https://togithub.com/go-shiori/shiori/pull/730)
- [@&#8203;cbe](https://togithub.com/cbe) made their first contribution
in
[https://github.com/go-shiori/shiori/pull/759](https://togithub.com/go-shiori/shiori/pull/759)
- [@&#8203;shirayu](https://togithub.com/shirayu) made their first
contribution in
[https://github.com/go-shiori/shiori/pull/756](https://togithub.com/go-shiori/shiori/pull/756)
- [@&#8203;JPFrancoia](https://togithub.com/JPFrancoia) made their first
contribution in
[https://github.com/go-shiori/shiori/pull/754](https://togithub.com/go-shiori/shiori/pull/754)
- [@&#8203;LLKoder](https://togithub.com/LLKoder) made their first
contribution in
[https://github.com/go-shiori/shiori/pull/794](https://togithub.com/go-shiori/shiori/pull/794)
- [@&#8203;istiak101](https://togithub.com/istiak101) made their first
contribution in
[https://github.com/go-shiori/shiori/pull/802](https://togithub.com/go-shiori/shiori/pull/802)
- [@&#8203;PterX](https://togithub.com/PterX) made their first
contribution in
[https://github.com/go-shiori/shiori/pull/836](https://togithub.com/go-shiori/shiori/pull/836)

**Full Changelog**:
go-shiori/shiori@v1.5.5...v1.6.0

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Renovate
Bot](https://togithub.com/renovatebot/renovate).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4yNTIuMCIsInVwZGF0ZWRJblZlciI6IjM3LjI1Mi4wIiwidGFyZ2V0QnJhbmNoIjoibWFzdGVyIn0=-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants