New in this version: new faster turbo triple mode, ham6 bayer 8×8 dithering, performance fixes and improvements.
Download:
Main mirror (all stable and testing versions)
External mirror
Changelog:
- EHB minimum enabled palette index selection (image will only use colors from this index – for overlay image on top on some effects/animations).
- HAM6/8 raw data map saving (together with a palette file can be used to recreate the image if iff file can’t be used for some reason).
- HAM6 turbo triple mode (faster version of triple mode checking less combinations). It should be 4 to 5 times faster with only small quality degradation. Available in the extra options tab and as a ham6_tt or ham6_normal_turbo cli mode parameter.
- HAM6: added bayer 8×8 dithering option (also available as a dither_bayer8x8 cli mode parameter that works in all modes). Useful for converting animations that normally exhibit strong banding in ham6 (tested with a modified agaconv).
- Fixed a bug slowing down conversion in cli or batch mode. Added conversion time info in cli mode.
- Improved task dividing in multi-threaded mode for more even difficulty of each part. Should improve triple and turbo triple mode conversion speed.
Great work! Thx!
Personally i get very nice results when converting 24 bit to 12 bit 16 color (Amiga Hires) with help of SColorQ (implemented by http://ximagic.com/q_index.html).
My workflow is something like this:
All image processing (like crop, resize etc) by Gimp
Launching Ximagic plugin where i usually reduce 24 bit to 12 bit using uniform RGB quantization (also possibility to use different dithering algorithms)
then once again Ximagic is used however with SColorQ selected (so dithering is disabled, 4 bits per pixel i.e. 16 color mode)
Usually produced image is grainy due the way how SColorQ works but created palette is in my opinion best so it can be exported and reused with other dithering algorithms usually with best visual results (once again – i can’t find better than SColorQ color quantization algorithm).
Good color palette selection (i.e. 16 CLUT) for HAM seem to be crucial as it may allow to create sharp transition (no HAM fringing).
This may lead to another idea for HAM processing – by using edge detection algorithm (horizontal only), search for best (optimal color from HAM i.e. fringing perspective) can be reduced to only areas where colors changing rapidly – this is where HAM suffer most from inability to change all RGB components at once.
For animation (but maybe not only) i can suggest Ulichney dithering – visually similar results to common error distribution ditherers (such as Floyd Steinberg) but free from common issues (less error propagation trough spatial-temporal domain) and as such similar to Bayer.
Curious to see Your comments on this.
I’ve tried scolorq. It produces great results but it’s quite slow even using an optimized c++ code. Java implementation will probably be too slow for most users, so for now you need to generate palette using ximagic or use build-in routines.
Using edge detection to optimize ham conversion is an interesting idea. I will do some research.
Regarding Ulichney dithering, I need to find more information to evaluate it. Wikipedia describes this method as quite slow: “It takes significant computational resources to generate the blue noise matrix: on a modern computer a 64×64 matrix requires a couple seconds using the original algorithm”.
Great – thx for reply!
Agree – scolorq is slow but still it can deliver very good CLUT when compared to regular mathematical approach (k-means, Heckbert etc).
Not sure how good can be Color Thief https://github.com/lokesh/color-thief (or its way faster incarnations https://github.com/SvenWoltmann/color-thief-java/ or https://github.com/soualid/java-colorthief).
Ulichney paper http://cv.ulichney.com/papers/1993-void-cluster.pdf and http://momentsingraphics.de/BlueNoise.html suggest to use precomputed matrix, especially tileable one can be efficient way to deal with computing complexity. So they can be embedded and reused multiple time.
Side to HAM – as common dithers usually propagate error in horizontal direction and HAM is sensitive also in only horizontal direction (as full change of the RGB pixel value require 3 pixels) then perhaps simple trick to rotate image by 90 deg before dither and rotate it backward after dithering may improve visual quality – regular ditherer as FS will propagate error mostly in vertical direction. Ideally dither for HAM should propagate error only in vertical direction so HAM fringing could be minimized (so perhaps normal dither can be modified to propagate error mostly in vertical direction).
Also i don’t recall source for another trick to speed up overall image processing time – perform processing on negated/inverted image.
In theory all processing shall be performed in linear color space so inverted gamma correction, processing then gamma correction reapplied but using negated non linear color space can partially nullify errors in processing.
Not sure if this can be feasible also in HAM.
With remark to my previous comment about error propagation and modification of the dither matrix itself (to change error propagation from horizontal to vertical) – perhaps scan order can be modified (instead serpentine in horizontal direction serpentine in vertical direction, and/or perhaps something more fancy like Peano order).
Btw with remark to palette change and interlace – there is no need to avoid color change each second line in interlace, limitation is in currently existing OS procedures where OS doesn’t distinguish between odd/even fields (this is itself quite bizzare as it prevent fully use interlace mode for example to perform smooth animations) however at the HW level long/short fields are supported and as such color can be change in every line – it is up to viewer to properly load color sequence.
Changing palette every second line reduces some artefacts. You can uncheck the interlace selector to change palette every line.
In HAM6 dithering that involves error propagation is applied during RGB888 to RGB444 conversion. Dithering during HAM operation calculation produced oversaturated colors, many visual glithes or was just broken. It was impossible to implement serpentine scanning and multi-threaded calculation. It was available in old ham_convert versions as a “oversaturate glitch mode” due to a undesired look, sometimes reminescent of the 1989 Batman museum vandalism scene.
HAM6 is “true” 12 bit RGB with Delta compression and only single component changed at once so IMHO no need to dither HAM – dithering is required for 24 bit to 12 bit conversion but later all 12 bit colors are available with some limitation – HAM fringes is typical case for Delta codec overload (due only single component can be changed in single step).
Issue with normal dither is that error is propagated with assumption of no Delta compression (i.e. no additional limits) that’s why HAM require different dither (aware of Delta compression).
btw this thread can be useful – some suggestion about unidirectional FS for example
https://forum.doom9.org/showthread.php?t=171766
Had some time so i tested ham_convert 1.8.4 and some problems are visible
firstly in EHB mode color selection is suboptimal – frequently dark colors (i.e. with level <8 are added to primary 32 colors – for example black and half black (i.e. bit 6) is still black – probably only bright colors should be counted in EHB mode first – perhaps you can apply some threshold function and mask everything bellow 128 to avoid adding such colors to primary half of CLUT.
PCHG / dynamic – i've noticed some issues with color selection – trying to use hybrid mode i.e. 16 colors, 11 colors static(locked) and remaining 5 freely selected.
I use ScolorQ to find best 11 colors palette then this palette is loaded, setting are 16 colors, 11 colors locked (my assumption already populated) and then 5 is freely selected based on general assumption that in lowres single color change take 8 pixels and in hires 16 pixels – obviously current algorithm doesn't work in such way.
Not sure if you how are doing color selection for PCHG but IMHO input picture can be downscaled by 8/16 (depends on lowres/hires) so some lowres color map can be created (it will be comparable to Copper chunky) then shared color map (in my case 11 colors from external palette) can be subtracted from this map and remaining colors used as entry for color quantization – such process should be quick as for 672×538 input (one i use – moderate overscan in hires lace) efficiently color map is 42×239 and for each line 5 colors can be selected. this should provide decent results. Of course some wise selection which color should be used (perhaps most distant from locked ones or most distant and most populated or some other method).
Dither algorithms – as i already proposed Ulichney (blue noise) then perhaps you may consider also very fast https://pippin.gimp.org/a_dither/
EHB palette color selection was always problematic and I’ve already tried this approach. Fixed threshold caused only very small improvement in bright areas, but lack of sufficient number of shades in the shadows was more noticeable. I’ve added numerous configuration settings like palette diversity (more varied colors including less dark colors), ehb animation mode, ehb extra palette optimization.
DynamicHires palette calculation prefers less line artefacting over changing as many colors as possible, but you can select use old DynamicHires error formula to get more colors. Checks and bayer dithering test how well various palettes mix to produce additional colors, that’s why they usually produce better results than lack of dithering.
Ulichney dithering and a_dither still require more work but so far Ulichney is not as good as bayer in ham6.
HAM as mode is tricky for dithering – normal dithering approach is to distribute quantization error with assumption that there is no additional errors but in HAM you have Delta compression and possibility to change only single component (R or G or B) at once so side to normal quantization error also Delta compression error is added. So quantization error need to be analyzed and shaped for at least 3 horizontal pixels – that’s why selection of the first 16 CLUT is crucial for overall conversion quality. Of course dithering in vertical direction is free from this limitations.
EHB explanation is sane but human eye is less sensitive for dark areas so even higher inaccuracies are less visible and when i tired many different pictures this was quite clear that frequently dark color was used in first 32 CLUT entries when it was quite clear that this color is quite comparable to second “shadow” CLUT entries (i refer to way how ham_convert present color palette).
Ulichney has many advantages over dither like FS, Sierra etc. It is static like Bayer but spectraly similar to FS.
So it may be beneficial for all animation conversion (perhaps not the case for RLL in common Amiga Anim but may be more tailored for some future video compressions codecs)
From some time i think to add additional HW to Amiga where HAM instead RGB use YCoCg space – it should be better from quality perspective, compression friendly and also give Amiga possibility to display more “natively” MPEG or H.264 compressed video (as MPEG1,2 and H.264 support YCoCg).