As a data project, pubkey/jsonschema-key-compression has picked up 103 stars on GitHub (TypeScript). Compress json-data based on its json-schema while still having valid json
Snapshot summary built from the project's own GitHub metadata — there's no written TopGit review yet. The page will update automatically when a full review is published.
WHY NO REVIEW YET
TopGit writes full reviews for the most-starred, most-requested repositories. This page is a snapshot until then — see the READ ME tab for the original README in full.
Compress json-data based on its json-schema while still having valid json.
It works by compressing long attribute-names into smaller ones and backwards.
The efficiency depends on the amount and length of the attribute names.
The uncompressed json-object from above has about 230 chars as string
With the key-compression, this can be reduced to 140 chars which saves about 40%
Just using gzip on the json would result in 180 chars
Using gzip+key-compression ends in a string with only 127 chars
The enum-compression saves additional space,
the longer the values of the enums are the more it saves.
You can reproduce these results by running npm run test:efficiency.
Performance
The compression works pretty fast. Here are some time measurements on a single CPU core of a Node.js 22 process. Only the measured operation is timed, the creation of the test-data is not part of the measurement.
Creating a compression-table from the schema of the object above takes about 0.005ms
Compressing the example-object from above takes about 0.0025ms
Decompressing takes about 0.002ms per object
You can reproduce these results by running npm run test:performance.
You should use this when
you want to save storage space in an NoSQL-database but still want to have valid queryable json-data
you transmit many objects in many small requests over the network so that gzip cannot be efficient
you want to store json-data inside of the browser-storage (indexedDB or localstorage) and you reach the storage limit
You should NOT use this when
you send many objects in a single request, you should rely on gzip instead
you do not want to still have valid json-data, you should use protobuf instead
you have schema-less data
Comparison with gzip
Gzip generates its compression-flags from the input. This makes it more efficient, the more data is compressed at once. But gzip is less efficient the smaller the dataset is.
The key-compression creates the compression-table from the jsonschema up front with has advantages when small pieces of data are compressed.
Usage
Install
npm install jsonschema-key-compression --save
createCompressionTable
Creates a compression-table from the json-schema.
import {
createCompressionTable,
DEFAULT_COMPRESSION_FLAG
} from 'jsonschema-key-compression';
const compressionTable = createCompressionTable(jsonSchema);
// all parameters
const compressionTable = createCompressionTable(
jsonSchema,
DEFAULT_COMPRESSION_FLAG, // the character that marks a compressed key
['myProperty'], // property-names that must not be compressed
true // enable the enum-compression, see below
);
Enum-compression
Enum-compression stores the value of a property as the index of that value
inside of the enum of the property. With the enum ['admin', 'editor', 'viewer']
the value 'editor' is stored as the number 1.
It is opt-in with the fourth parameter of createCompressionTable(),
so that data which was compressed without it stays readable.
A property is only enum-compressed when it is safe to do so:
All values of the enum must be strings. A number inside of the enum could not be
told apart from an index, and strings are where the saved space is.
The property name must have the same enum everywhere in the schema.
A name that is used with different enums, or that has no enum somewhere else,
is left alone because the same number would otherwise mean different things.
Properties in ignoreProperties are not enum-compressed.
The enum-values are sorted before the indexes are assigned, so the order of the indexes
is the alphabetical order of the values. Because of this, sort and the range-operators
$gt, $gte, $lt and $lte keep working on compressed data.
compressQuery() also transforms the values of $eq, $ne, $in, $nin, $all,
$not and $elemMatch, and resolves a $regex into the $in of the matching indexes.
The operators $type and $mod are not transformed, they would run on the
number of the compressed value.
Enum-compression assumes that the data matches the schema.
A document that stores a number where the schema defines an enum of strings
cannot be told apart from an already compressed value.
Does pubkey/jsonschema-key-compression have a project website?
No homepage URL was recorded for pubkey/jsonschema-key-compression in TopGit's last sync. The README tab above frequently contains screenshots and demo links, or check the repository description on GitHub.
How active is development on pubkey/jsonschema-key-compression?
The most recent commit recorded on pubkey/jsonschema-key-compression was 11 days ago, based on the GitHub push timestamp. The repository has 11 forks — one of the better signals of community interest.
How many stars does pubkey/jsonschema-key-compression have?
pubkey/jsonschema-key-compression has 103 GitHub stars — refresh the page for the live number, or check github.com/pubkey/jsonschema-key-compression. TopGit mirrors GitHub's count but does not claim minute-by-minute accuracy.
What language is pubkey/jsonschema-key-compression written in?
pubkey/jsonschema-key-compression is written primarily in TypeScript. GitHub's language field is based on the largest share of bytes in the default branch.
What license does pubkey/jsonschema-key-compression use?
pubkey/jsonschema-key-compression is released under the Apache-2.0 license. Always verify the LICENSE file directly on GitHub for the authoritative terms — license strings can be edited out of sync with a project's actual stance.
What topics is pubkey/jsonschema-key-compression associated with?
GitHub's repository topics for pubkey/jsonschema-key-compression: "compression", "json", "json-schema", "jsonschema", "nosql". TopGit's editorial category is Data.
Where do I read more about pubkey/jsonschema-key-compression?
This TopGit page is a snapshot — the READ ME tab shows the project's own README content (links stripped, images preserved). The GitHub repository at github.com/pubkey/jsonschema-key-compression is the definitive source.
Read full README in the tab above.
Want a second opinion on jsonschema-key-compression?
Ask an AI that can read this page — one click and you get its take on jsonschema-key-compression.