APNG support on main libpng library?

[LocutusOfBorg]

Hello, I don't know why the apng project is kept separate, I see arch-linux is adding the patch manually, and some people asked me to do the same in Debian...
do you have any good reason for not merging the patch into the main repo?
https://sourceforge.net/projects/libpng-apng/

thanks

[LocutusOfBorg]

Hello, ping?

[svgeesus]

Indeed, it is suboptimal to keep PNG and APNG separate.

The original reason was that APNG failed the vote at the PNG group. Since then, though, APNG has become widely supported so in my opinion it makes sense to add to the main PNG specification and also to roll in the patches to libpng. I did suggest discussion and a re-vote, in Sept 2017 but there was little response.

[ctruta]

I guess it could work, if somebody would bring it to our lists and drive the Call For Discussion / Call For Vote, and also advocate it if necessary.

If implementation and maintenance of a libpng patch is the only problem, then the necessity of a patch (as opposed to a properly-implemented set of callbacks) is a libpng design problem that needs fixing.

If that's indeed the case, then I guess I can say it's the biggest libpng design problem. Then it is libpng (not PNG) that truly needs fixing.

[LocutusOfBorg]

@ctruta can you please do it? I admit I didn't even understand your last comment...

[svgeesus]

@ctruta wrote:

I guess it could work, if somebody would bring it to our lists and drive the Call For Discussion / Call For Vote, and also advocate it if necessary.

I tried that some years ago, but there was no real discussion and, like most proposals over the last few years, nothing happened.

Meanwhile the PNG specification is badly needing maintenance. There are many errata reported since 2003. To help with that, I forked the W3C PNG specification to make an Editors Draft. Each reported erratum is raised as a GitHub issue and the clear and obvious ones I have rolled into the Editors Draft so the changes can be viewed in place. The others need to be discussed.

I don't think the original PNG mailing list is the right way to handle specification maintenance anymore. The discussion procedure is baroque, there is a poor record of discussing or adopting proposals, and momentum has ground to a halt. I think an open W3C Working Group is the better way to handle this.

Beyond just maintenance though, there is a clear need to re-align the PNG specification with widely-implemented reality, especially as regards APNG.

An updated PNG Specification which incorporates APNG would then allow libpng to incorporate support directly, rather than a plethora of ad-hoc patches, forks, or other means to support it.

Cosmin, I would really value your comments on this path forwards. Continued alignment between the PNG specification and libpng is very important. libpng is being held back by the brokenness of the current PNG spec maintenance process.

[benkasminbullock]

The PNG mailing list is the sourceforge libpng mailing list? It seems very inactive but it might be a good idea to at least post this here. In a recent discussion one or two members of that mailing list did not know/were surprised that development of libpng is continuing under the late Glenn Randers-Pehrson's github account. Also Cosmin Truta seems not to have very much time to devote at the moment, so it might be good if there was at least one other member with ability to commit here.

[svgeesus]

Do you have a pointer to the recent discussion? Last relevant mail about libpng which I see in the PNG Group archives is 2019-06-18 05:21:38 from Cosmin.

Ah! You mean png-mng-implement (at) lists.sourceforge.net. I'm not subscribed but sure, it makes sense to post there too.

Yes, I see the discussion you mentioned

[svgeesus]

It seems very inactive but it might be a good idea to at least post this here.

Good idea, done.

[svgeesus]

APNG in the PNG specification, third edition

[nanonyme]

APNG in the PNG specification, third edition

What does this mean, exactly? Does it mean that if someone now submitted APNG support into libpng, there would be high probability it would be actually accepted?

[jbowler]

There are two proven approaches to APNG support and they are mostly incompatible. The first, written by Mozilla but maybe no long used, modifies the distributed libpng library, the second uses the distributed library as-is and handles APNG via the existing "unknown" chunk mechanism.

The Mozilla approach broke other things:

https://bugs.gentoo.org/824834

The issues are discussed at length here:

https://bugs.gentoo.org/824018

To copy'n'paste two (separate) lines from #824018:

Chromium has its own decoder for APNG on top of libpng and they don't require libpng patching.

Apple demonstrated in Webkit to implement a decoder for APNG without patching libpng.

It isn't clear to me that either approach is likely to interact correctly with the existing libpng API. For example if I write a palette (colortype 3) APNG and the animation frames have alpha (tRNS) values which are neither 0 nor 1(255) the result is not computable without specific transformations nevertheless this APNG is completely permitted by the specification (unless I'm missing something). See:

https://www.w3.org/TR/png-3/ figure 13:

I see no restriction on fdAT or fcTL with regard to colortype 3 (but as I said I might be missing something).

The simple answer to all this is to provide libapng, a separate library, which implements an APNG reader with a much more limited API. In particular it would probably only provide 8 or 16-bit per component output; somewhat like the Simplified API but without any palette format.

This has the additional advantage that currently applications which handle static images would not suffer code bloat from something else they don't use. libpng is already well bloated.

Since both Google and Apple have done it this way (using, I assume, unknown chunk handling) and since this has probably defined a "minimal" support API this approach seems compelling.

The alternative would have to work in the presence of all the libpng transform APIs or it would have to do exactly the same thing as my hypothetical libpng; implement yet-another top level API which restricts the available transforms and output formats (in the same manner as the addition of the Simplified API).

In any case there is the issue of authoring APNG files; libpng both reads and writes PNG files, so if chunks are supported the write side needs support too. I don't know how well the existing implementations support the write side but it strikes me as potentially very messy; clearly APNG authoring programs exist but maybe they don't use libpng.

[ProgramMax]

I'm not yet familiar with libpng's transformations.
Do you mean decoding a full frame 2 requires frame 1? If so, transparency doesn't need to complicate it, right? It could be a 100x100 image with a frame 2 updating a 10x10 region.
Or do you mean to pre-blend the pixels of that 10x10 region, which still requires frame 1? If so, libpng could provide the unblended version, right?

I'm trying to follow so I understand the problem. My concern is if this is a spec problem. If so, we can change the spec.

[vrubleg]

As far as I remember from old discussions when it was implemented in Chromium, blending is done manually, and there are test images that cover such things: http://littlesvr.ca/apng/test.html

[ProgramMax]

That is correct. Chromium currently treats each frame as full.
So if frame 2 only updates a 10x10 region, it'll first copy frame 1 into frame 2, then update that region (performing the blending it needs). (I'm glossing over the disposal method here for simplicity.)

[jbowler]

I'm not yet familiar with libpng's transformations.

I mean the set of APIs implemented in pngrtran.c (for APNG read), pngwtran.c (for APNG write) and pngtrans.c (for both). Doing a quick grep there are 35 APIs in there but a few are support APIs.

If the libpng user (app or library) can get hold of the png_struct then it is to be expected that those APIs will be invoked. Despite my earlier misgivings I've worked out an implementation inside libpng itself which leverages the transforms to allow the user to get the necessary result regardless of what it is.

libpng already defines the behavior of these transforms and there is no need for invention here because the two possibilites of extracting the subframe with alpha or composing the subframe into the rows supplied by the user are adequate for APNG.

I also have an algorithm for defining colortype 3 compostion which is really only likely for GIFs with a single transparent color; 0<alpha<1 can be handled by halftoning.

I'm also pretty sure that this can be done without breaking the existing API; breaking the API isn't an option, so that means preserving the default "unknown" chunk handling. It turns out that this just works when done correctly and the current unknown chunk handling seems entirely sufficient to address the major issue of having to buffer the entire stream under some circumstances.

In addition I think it is even possible to do this in a minor release; that requires ABI downward compatibility (existing apps/libraries need not be recompiled) as well as complete API compatibility (excepting previously deprecated APIs).

Nevertheless the final decision is Cosmin's and there is a strong argument for a major release because major releases can coexist on the same machine. E.g. I can have libpng 1.0, 1.2, 1.4, 1.5 and 1.6 all installed at the same time just like I do, at present, have Qt5 and Qt6 both installed.

If so, libpng could provide the unblended version, right?

That's a minimal requirement of PNG editors; they need the original data, however my point above is that both options are fully supported at present along with all the transformations required to convert to a common format (e.g. 8-bit RGBA). In other words the code is already there.

My concern is if this is a spec problem. If so, we can change the spec.

This is a decoder issue, the decoder, which is libpng plus however the app/library uses it, can do what it wants. The spec only recommends (like gamma correction, like doing alpha composition correctly, like ignoring cHRM or iCCP or cICP). How many apps actually support cHRM? It's easy - libpng provides a function to return the corresponding CIEXYZ triple - but how many apps do any colour correction?

One this is certain; libpng supports alpha composition (though I do need to check that APNG_BLEND_OP_SOURCE is possible) but not with colour correction; gAMA handling happens, but cHRM, iCCP and cICP will never work inside libpng without a major rewrite (floating point format support). That includes the encoding transforms in iCCP and cICP both of which can generate values outside the range 0..1, particularly the absolute values in cICP (I don't know if that is possible in an ICC profile, but I don't care since libpng provides no support for iCCP beyond the decompression and basic checks; like eXIf.)

Spec concerns I have are with regard to tight definition of the format and I've yet to find a spec issue in the APNG stuff. There are a couple of things that may benefit from clarification though I think they're pretty clear:

1. What happens when the last fCTL has a dispose_op of APNG_DISPOSE_OP_NONE or PREVIOUS and aCTL num_plays is >1. [Conclusion: exactly what it says; the display of the frame with sequence number 0 uses the disposal method of the last frame the after the first time round and, as explicitly defined, APNG_DISPOSE_OP_BACKGROUND the first time.]
2. What happens when an APNG has no alpha or transparent color (so the static image and frames are completely opaque) and one or more frames have fCTL chunks that specify a sub-frame with APNG_DISPOSE_OP_BACKGROUND. [Conclusion: scan all the fCTL chunks first to check for this case or require the canvas to have an alpha channel or transparent color, or just use a fixed background color - it's an app decision.]

@vrubleg:

As far as I remember from old discussions when it was implemented in Chromium, blending is done manually, and there are test images that cover such things: http://littlesvr.ca/apng/test.html

Would it be possible for those to be licensed/unlicensed/public-domained so that libpng can include them in releases (source tarballs)? Without test images we can't write unit tests and the lack of unit tests has been a real source of bad bugs over the years.

With regard to blending all possibilities seem to have the required coverage in libpng. The API means that the app has to handle the placement (but not actual composition) of a frame within the canvas which means the app has to handle the gnarly problem of frames with sub-pixel alignment (low bit gray and palette). libpng never had any meaningful support for oFFs.

So far as a high level API is concerned I would expect it to behave like the Simplified API and simplify! Make it so that frames were expanded to a small number of formats (fewer than the Simplified API, maybe just RGBA and GA) and handle the compose/dispose stuff internally, so the output frames can be displayed without any more work (other than composition onto the background if it's not fixed.)

I took a quick look at the Rust png crate but it doesn't seem to document what next_frame returns; the result could be either. Seems to be UTSL or experiment (suck it and see.)

[jbowler]

In view of @ctruta's lack of comment since Chris's apparent declaration of a putsch on April 29, 2021 I've abandoned the direct modification approach. It doesn't buy anything but unending pain:

Cosmin has to accept it, but where? 1.6.44 was, as I understand it, a final clean-up so would everyone have to wait for libpng2, or libpng-ng?

A separate library would use the Google/Apple approach (if I understand it; I deliberately haven't read the code, I have no desire to be sued). It would work with most versions of libpng (1.6 but not just 1.6) it wouldn't break any API on the planet (obviously, at least I hope that's obvious) and, if Cosmin accepts the small add of a sub-project or sub-directory, it would be part of libpng.

[vrubleg]

@jbowler

Would it be possible for those to be licensed/unlicensed/public-domained so that libpng can include them in releases (source tarballs)?

I tried to reach out to the author of the test, but he doesn't reply for some reason.

The test is already used by the WebKit project at least. It was proposed here by its creator and merged afterwards, all the test files are in the WebKit repository. So I think it should be fine to use the files in this project too.

[jbowler]

I'm going to have to build my own test set; the delays in those images are too small to see the intermediate bugs (possibly WebKit test is using automatic capture and binary compare for unit tests.) I can alter timings, blend and disposal modes with Jason Summer's TweakPNG but the current version doesn't seem to support addition of an fDAT (e.g. by importing a PNG file).

The other things I found on the web are what O would call converters - apngasm for example seems to take full frames and then make its own choice of how to optimally represent them. There used to be a png->text->png set of converters but I can't find that code any more.

It seems it's time for a good basic assembly/disassembly tool. The basic functionality for one is also a library requirement; editors need to be able to extract individual uncomposed frames

[maxstepin]

Yes, you can use the test set http://littlesvr.ca/apng/test.html whatever you like,
I used such small delays, because it was designed for automated testing, but adjusting them is not a problem.
I'll dig up those converters you mentioned, let me know what kind of thing you have in mind.

[jbowler]

let me know what kind of thing you have in mind.

I sort-of remember something called pngdump which would output a textual representation of a PNG. I was thinking of writing something which would do that but then be able to read it back in and produce the corresponding (binary) PNG. I have to admit I've completely lost enthusiasm for doing anything at this point; the W3C seems to have decided to do nothing (just document the current private chunks, which are a done deal).

[qbnu]

Some other test files (zip):
https://philip.html5.org/tests/apng/tests.html
https://commons.wikimedia.org/wiki/Category:Animated_PNG_files
https://web.archive.org/web/20090713004416/http://people.mozilla.com/~dolske/apng/
https://web.archive.org/web/20101025122604/http://animatedpng.com/index.php/samples/apng-gallery/

[peterkaczorowski]

Here is libpng 1.6.44 patched with APNG support:
https://github.com/peterkaczorowski/libpng/tree/v1.6.44-apng

and the demos:

• https://github.com/peterkaczorowski/libpng_apng_read_demo - show two layers of APNG file
• https://github.com/peterkaczorowski/libpng_apng_write_demo - makes APNG file from two PNG files.

[peterkaczorowski]

Here is libpng 1.6.45 patched with APNG support:
https://github.com/peterkaczorowski/libpng/tree/v1.6.45-apng

[peterkaczorowski]

Another is libpng 1.6.45 patched with APNG support with extra files fot wxWidgets library:
https://github.com/peterkaczorowski/libpng/tree/v1.6.45-apng-wxwidgets