Indexed by TopGit from live GitHub metadata: openzipkin/zipkin-dependencies has 179 stars, written primarily in Java. Spark job that aggregates zipkin spans for use in the UI
Snapshot summary built from the project's own GitHub metadata — there's no written TopGit review yet. The page will update automatically when a full review is published.
WHY NO REVIEW YET
TopGit writes full reviews for the most-starred, most-requested repositories. This page is a snapshot until then — see the READ ME tab for the original README in full.
Zipkin Dependencies collects spans from storage, analyzes links between services, and stores them for later presentation in the web UI (ex. http://localhost:8080/dependency).
This process is implemented as an Apache Spark job. This job parses all traces in the current day in UTC time. This means you should schedule it to run just prior to midnight UTC.
All Zipkin Storage Components
are supported, including Cassandra, MySQL and Elasticsearch.
Versions
STORAGE_TYPE=cassandra3 : requires Cassandra 3.11.3+; tested against the latest patch of 4.0
STORAGE_TYPE=mysql : requires MySQL 5.6+; tested against MySQL 10.11
STORAGE_TYPE=elasticsearch : requires Elasticsearch 7+ or OpenSearch 2.x; tested against last minor release of Elasticsearch 7.x and 8.x, OpenSearch 2.x
Quick-start
Until SPARK-43831, Zipkin Dependencies requires Java 17 to run.
The quickest way to get started is to fetch the latest released job as a self-contained jar. For example:
You can also start Zipkin Dependencies via Docker.
$ docker run --env STORAGE_TYPE=cassandra3 --env CASSANDRA_CONTACT_POINTS=host1,host2 openzipkin/zipkin-dependencies
For a more complete example, take a look at the docker-compose example.
Usage
By default, this job parses all traces since midnight UTC. You can parse traces for a different day
via an argument in YYYY-mm-dd format, like 2016-07-16.
# ex to run the job to process yesterday's traces on OS/X
$ STORAGE_TYPE=cassandra3 java -jar zipkin-dependencies.jar `date -uv-1d +%F`
# or on Linux
$ STORAGE_TYPE=cassandra3 java -jar zipkin-dependencies.jar `date -u -d '1 day ago' +%F`
Environment Variables
zipkin-dependencies applies configuration parameters through environment variables.
The following variables are common to all storage layers:
* `SPARK_MASTER`: Spark master to submit the job to; Defaults to `local[*]`
* `ZIPKIN_LOG_LEVEL`: Log level for zipkin-related status; Defaults to INFO (use DEBUG for details)
* `SPARK_CONF`: Extend more spark configuration with value in properties format and separated with comma. Such as `spark.executor.heartbeatInterval=600000,spark.network.timeout=600000`
Cassandra
Cassandra is used when STORAGE_TYPE=cassandra3. The schema is compatible with Zipkin's Cassandra storage component.
Here are the variables that apply
* `CASSANDRA_KEYSPACE`: The keyspace to use. Defaults to "zipkin".
* `CASSANDRA_CONTACT_POINTS`: Comma separated list of hosts / ip addresses part of Cassandra cluster. Defaults to localhost
* `CASSANDRA_LOCAL_DC`: The local DC to connect to (other nodes will be ignored)
* `CASSANDRA_USERNAME` and `CASSANDRA_PASSWORD`: Cassandra authentication. Will throw an exception on startup if authentication fails
* `CASSANDRA_USE_SSL`: Requires `javax.net.ssl.trustStore` and `javax.net.ssl.trustStorePassword`, defaults to false.
* `STRICT_TRACE_ID`: When false, dependency linking only looks at 64 bits of a trace ID, defaults to true.
MySQL is used when STORAGE_TYPE=mysql. The schema is compatible with Zipkin's MySQL storage component.
* `MYSQL_DB`: The database to use. Defaults to "zipkin".
* `MYSQL_USER` and `MYSQL_PASS`: MySQL authentication, which defaults to empty string.
* `MYSQL_HOST`: Defaults to localhost
* `MYSQL_TCP_PORT`: Defaults to 3306
* `MYSQL_USE_SSL`: Requires `javax.net.ssl.trustStore` and `javax.net.ssl.trustStorePassword`, defaults to false.
Elasticsearch/OpenSearch is used when STORAGE_TYPE=elasticsearch. The schema is compatible with Zipkin's Elasticsearch storage component.
* `ES_INDEX`: The index prefix to use when generating daily index names. Defaults to zipkin.
* `ES_DATE_SEPARATOR`: The separator used when generating dates in index.
Defaults to '-' so the queried index look like zipkin-yyyy-DD-mm
Could for example be changed to '.' to give zipkin-yyyy.MM.dd
* `ES_HOSTS`: A comma separated list of Elasticsearch / OpenSearch hosts advertising http. Defaults to
localhost. Add port section if not listening on port 9200. Only one of these hosts
needs to be available to fetch the remaining nodes in the cluster. It is
recommended to set this to all the master nodes of the cluster. Use url format for
SSL. For example, "https://yourhost:8888"
* `ES_NODES_WAN_ONLY`: Set to true to only use the values set in ES_HOSTS, for example if your
Elasticsearch / OpenSearch cluster is in Docker. Defaults to false
* `ES_USERNAME` and `ES_PASSWORD`: Elasticsearch / OpenSearch basic authentication. Use when X-Pack security
(formerly Shield) is in place. By default no username or
password is provided to elasticsearch.
Example usage:
$ STORAGE_TYPE=elasticsearch ES_HOSTS=host1,host2 java -jar zipkin-dependencies.jar
# To override the http port, add it to the host string
$ STORAGE_TYPE=elasticsearch ES_HOSTS=host1:9201 java -jar zipkin-dependencies.jar
Custom certificates
When using an https endpoint in ES_HOSTS, you can use the following standard properties to
customize the certificates used for the connection:
javax.net.ssl.keyStore
javax.net.ssl.keyStorePassword
javax.net.ssl.trustStore
javax.net.ssl.trustStorePassword
Building locally
To build the job from source and run against a local cassandra, in Spark's standalone mode.
# Build the spark jobs
$ ./mvnw -T1C -q --batch-mode -DskipTests -Denforcer.fail=false package
$ STORAGE_TYPE=cassandra java -jar ./main/target/zipkin-dependencies*.jar
Running in a Spark cluster
The jar file produced by this build can also run against spark directly. Before anything
else, make sure you are running the same version of spark as used here.
You can use the following command to display what this project is built against:
Note that the Zipkin team focuses on tracing, not Spark support. If you have Spark cluster related
troubleshooting questions, please use their support tools.
Troubleshooting
When troubleshooting, always set ZIPKIN_LOG_LEVEL=DEBUG as this output
is important when figuring out why a trace didn't result in a link.
If you set SPARK_MASTER to something besides local, remember that log
output also ends up in stderr of the workers.
By default, this job uses the value of system property java.io.tmpdir as location to store temporary data.
If you're getting java.io.IOException: No space left on device while processing large sets
of trace data, you can specify a different location that has enough space available using -Djava.io.tmpdir=/other/location.
Artifacts
All artifacts publish to the group ID "io.zipkin.dependencies". We use a common
release version for all components.
Library Releases
Releases are at Maven Central
Library Snapshots
Snapshots are uploaded to Sonatype after
commits to master.
Docker Images
Released versions of zipkin-dependencies are published to Docker Hub as openzipkin/zipkin-dependencies
and GitHub Container Registry as ghcr.io/openzipkin/zipkin-dependencies.
Does openzipkin/zipkin-dependencies have a project website?
No homepage URL was recorded for openzipkin/zipkin-dependencies in TopGit's last sync. The README tab above frequently contains screenshots and demo links, or check the repository description on GitHub.
How active is development on openzipkin/zipkin-dependencies?
The most recent commit recorded on openzipkin/zipkin-dependencies was 3 months ago, based on the GitHub push timestamp. The repository has 79 forks — one of the better signals of community interest.
Is openzipkin/zipkin-dependencies open source?
Yes — openzipkin/zipkin-dependencies ships under the Apache-2.0 license, which makes its source code freely readable (and, depending on license terms, forkable and reusable). Source: github.com/openzipkin/zipkin-dependencies.
What license does openzipkin/zipkin-dependencies use?
openzipkin/zipkin-dependencies is released under the Apache-2.0 license. Always verify the LICENSE file directly on GitHub for the authoritative terms — license strings can be edited out of sync with a project's actual stance.
What topics is openzipkin/zipkin-dependencies associated with?
GitHub's repository topics for openzipkin/zipkin-dependencies: "openzipkin". TopGit's editorial category is open-source.
Where do I read more about openzipkin/zipkin-dependencies?
This TopGit page is a snapshot — the READ ME tab shows the project's own README content (links stripped, images preserved). The GitHub repository at github.com/openzipkin/zipkin-dependencies is the definitive source.
Read full README in the tab above.
Curious whether zipkin-dependencies is right for you?
Let ChatGPT, Claude, or Perplexity look into it — click below and see what AI actually says about zipkin-dependencies.