Krita/Tile Data Format

Reasoning

After some tests made by Cyrille [1], it turned out that saving of file is very slow in Krita. The cause of it was KZip compression of all the tiles of the image. It was very slow in comparison with, e.g. LZF[2] or LZO[3]. So we decided to optimize it. Disabling KZip and including other algorithms into KoStore is a difficult task, so we decided to compress every tile in the tile engine.

Implementation in the tile engine

When discussing the details we decided to make the compression code shared between save-subsystem and swapper subsystem. So what is decided:

Any particular compression algorith will inherit KisAbstractCompression class:

class KisAbstractCompression {
public:
    // return false when output buffer is not enough
    virtual bool compress(void* data, qint32 data_length, void* output, qint32 output_length) = 0;
    virtual bool decompress(void* data, int32 data_length, void* output, qint32 output_length) = 0;
     
    // for some algorithms, those decide to be optimized to the data size
    virtual void adjustForDataSize(qint32 dataSize) { } 
};

This class will encapsulate lowlevel compression code.

The higher level of abstraction will encasulate loading of one tile to/from a buffer. This abstraction is done by KisAbstractTileCompressor class. This class will use a particular compression algorithm for for compression the data and writing a header into the same buffer (the header will be uncompressed). And this very class will be used to support different formats of tiles caused by backward compatibility.

class KisAbstractTileCompressor {
public:
    // returns number of bytes writen to the buffer with out-parameter
    virtual void compressTile(KisTileSP tile, void* buffer, qint32 &bytesWritten) = 0;
    
    // returns number of bytes read from the buffer using out-parameter
    // mm - memento manager, used for creating a new tile
    // used by swapper
    virtual KisTileSP decompressTile(KisMementoManager *mm, void* buffer, qint32 &bytesRead) = 0;
     
    // returns number of bytes read from the buffer using out-parameter
    // ht - hash table, that is used as a factory for lazy creation of tiles
    // used by KisTiledDataManager::read()
    virtual void decompressTile(KisTileHashTable *ht, void* buffer, qint32 &bytesRead) = 0;
    
    // returns the (size of a tile) + (size of a header)
    virtual qint32 maxBufferSize() = 0;
};

By the end of the change we will have at least two childs of this class: class for legacy 2.2 tiles, and one more for compressed 2.3 tiles.

Tile data format

VERSION 2
TILEWIDTH 64
TILEHEIGHT 64
PIXELSIZE 4
TILESNUMBER 10
DATA
0,0,LZF,12314
[12314 bytes of binarydata]
64,64,NONE,3333
[3333 bytes of binarydata]

First five lines are written by datamanger. Then the tiles go in format: x,y,<compression name>,size\n<binary data>

References

[1] - [http://lists.kde.org/?l=kde-kimageshop&m=126877404917830&w=2]

[2] - [http://liblzf.plan9.de/]

[3] - [http://www.oberhumer.com/opensource/lzo/]