GITHUB SNAPSHOT
josephmisiti/awesome-public-datasets
J
QUICK VERDICT
josephmisiti/awesome-public-datasets is a open-source project on GitHub. It has 13 stars. An awesome list of high-quality open datasets in public domains (on-going).
Snapshot summary built from the project's own GitHub metadata — there's no written TopGit review yet. The page will update automatically when a full review is published.
WHY NO REVIEW YET
TopGit writes full reviews for the most-starred, most-requested repositories. This page is a snapshot until then — see the READ ME tab for the original README in full.
Snapshot
Top contributors
Show top contributors
Awesome Public Datasets
.. image:: https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg :alt: Awesome :target: https://github.com/sindresorhus/awesome .. image:: https://travis-ci.org/caesar0301/awesome-public-datasets.svg :target: https://travis-ci.org/caesar0301/awesome-public-datasets
This list of public data sources <https://github.com/caesar0301/awesome-public-datasets>_
are collected and tidied from blogs, answers, and user responses.
Most of the data sets listed below are free, however, some are not.
Other amazingly awesome lists can be found in the
awesome-awesomeness <https://github.com/bayandin/awesome-awesomeness>_ and
sindresorhus's awesome <https://github.com/sindresorhus/awesome>_ list.
Contents
.. contents::
Agriculture
U.S. Department of Agriculture's PLANTS Database <http://www.plants.usda.gov/dl_all.html>_
Biology
1000 Genomes <http://www.1000genomes.org/data>_American Gut (Microbiome Project) <https://github.com/biocore/American-Gut>_Collaborative Research in Computational Neuroscience (CRCNS) <http://crcns.org/data-sets>_EBI ArrayExrepss <http://www.ebi.ac.uk/arrayexpress/>_ENCODE project <https://www.encodeproject.org>_Ensembl Genomes <http://ensemblgenomes.org/info/genomes>_Gene Expression Omnibus (GEO) <http://www.ncbi.nlm.nih.gov/geo/>_Gene Ontology (GO) <http://geneontology.org/page/download-annotations>_Global Biotic Interations (GloBI) <https://github.com/jhpoelen/eol-globi-data/wiki#accessing-species-interaction-data>_Human Microbiome Project (HMP) <http://www.hmpdacc.org/reference_genomes/reference_genomes.php>_ICOS PSP Benchmark <http://ico2s.org/datasets/psp_benchmark.html>_MIT Cancer Genomics Data <http://www.broadinstitute.org/cgi-bin/cancer/datasets.cgi>_NIH Microarray data <http://bit.do/VVW6>_ orFTP <ftp://ftp.ncbi.nih.gov/pub/geo/DATA/supplementary/series/GSE6532/>_OpenSNP genotypes data <https://opensnp.org/>_Pathguid - Protein-Protein Interactions Catalog <http://www.pathguide.org/>_Protein Data Bank <http://www.rcsb.org/>_PubChem Project <https://pubchem.ncbi.nlm.nih.gov/>_PubGene (now Coremine Medical) <http://www.pubgene.org/>_Sequence Read Archive(SRA) <http://www.ncbi.nlm.nih.gov/Traces/sra/>_Stanford Microarray Data <http://smd.stanford.edu/>_The Catalogue of Life <http://www.catalogueoflife.org/content/annual-checklist-archive>_The Personal Genome Project <http://www.personalgenomes.org/>_ orPGP <https://my.pgp-hms.org/public_genetic_data>_UCSC Public Data <http://hgdownload.soe.ucsc.edu/downloads.html>_UniGene <http://www.ncbi.nlm.nih.gov/unigene>_
Climate/Weather
Australian Weather <http://www.bom.gov.au/climate/dwo/>_Brazilian Weather - Historical data (In Portuguese) <http://sinda.crn2.inpe.br/PCD/SITE/novo/site/>_Canadian Meteorological Centre <http://weather.gc.ca/grib/index_e.html>_Climate Data from UEA (updated monthly) <https://crudata.uea.ac.uk/cru/data/temperature/#datter and ftp://ftp.cmdl.noaa.gov/>_European Climate Assessment & Dataset <http://eca.knmi.nl/>_Global Climate Data Since 1929 <http://en.tutiempo.net/climate>_NASA Global Imagery Browse Services <https://wiki.earthdata.nasa.gov/display/GIBS>_NOAA Bering Sea Climate <http://www.beringclimate.noaa.gov/>_NOAA Climate Datasets <http://www.ncdc.noaa.gov/data-access/quick-links>_NOAA Realtime Weather Models <http://www.ncdc.noaa.gov/data-access/model-data/model-datasets/numerical-weather-prediction>_The World Bank Open Data Resources for Climate Change <http://data.worldbank.org/developers/climate-data-api>_UEA Climatic Research Unit <http://www.cru.uea.ac.uk/data>_WorldClim - Global Climate Data <http://www.worldclim.org>_WU Historical Weather Worldwide <https://www.wunderground.com/history/index.html>_
Complex Networks
CrossRef DOI URLs <https://archive.org/details/doi-urls>_DBLP Citation dataset <https://kdl.cs.umass.edu/display/public/DBLP>_NBER Patent Citations <http://nber.org/patents/>_NIST complex networks data collection <http://math.nist.gov/~RPozo/complex_datasets.html>_Protein-protein interaction network <http://vlado.fmf.uni-lj.si/pub/networks/data/bio/Yeast/Yeast.htm>_PyPI and Maven Dependency Network <https://ogirardot.wordpress.com/2013/01/31/sharing-pypimaven-dependency-data/>_Scopus Citation Database <https://www.elsevier.com/solutions/scopus>_Small Network Data <http://www-personal.umich.edu/~mejn/netdata/>_Stanford GraphBase (Steven Skiena) <http://www3.cs.stonybrook.edu/~algorith/implement/graphbase/implement.shtml>_Stanford Large Network Dataset Collection <http://snap.stanford.edu/data/>_The Koblenz Network Collection <http://konect.uni-koblenz.de/>_The Laboratory for Web Algorithmics (UNIMI) <http://law.di.unimi.it/datasets.php>_The Nexus Network Repository <http://nexus.igraph.org/>_UCI Network Data Repository <https://networkdata.ics.uci.edu/resources.php>_UFL sparse matrix collection <http://www.cise.ufl.edu/research/sparse/matrices/>_WSU Graph Database <http://www.eecs.wsu.edu/mgd/gdb.html>_Stanford Longitudnal Network Data Sources <http://stanford.edu/group/sonia/dataSources/index.html>_
Computer Networks
3.5B Web Pages from CommonCraw 2012 <http://www.bigdatanews.com/profiles/blogs/big-data-set-3-5-billion-web-pages-made-available-for-all-of-us>_53.5B Web clicks of 100K users in Indiana Univ. <http://cnets.indiana.edu/groups/nan/webtraffic/click-dataset/>_CAIDA Internet Datasets <http://www.caida.org/data/overview/>_ClueWeb09 - 1B web pages <http://lemurproject.org/clueweb09/>_ClueWeb12 - 733M web pages <http://lemurproject.org/clueweb12/>_CommonCrawl Web Data over 7 years <http://commoncrawl.org/the-data/get-started/>_CRAWDAD Wireless datasets from Dartmouth Univ. <https://crawdad.cs.dartmouth.edu/>_Criteo click-through data <http://labs.criteo.com/2015/03/criteo-releases-its-new-dataset/>_Open Mobile Data by MobiPerf <https://console.developers.google.com/storage/openmobiledata_public/>_UCSD Network Telescope, IPv4 /8 net <http://www.caida.org/projects/network_telescope/>_
Contextual Data
Context-aware data sets from five domains <http://students.depaul.edu/~yzheng8/DataSets.html#Data>_ orGitHub <https://github.com/irecsys/CARSKit/tree/master/context-aware_data_sets>_
Data Challenges
Challenges in Machine Learning <http://www.chalearn.org/>_CrowdANALYTIX dataX <http://data.crowdanalytix.com>_D4D Challenge of Orange <http://www.d4d.orange.com/en/home>_DrivenData Competitions for Social Good <http://www.drivendata.org/>_ICWSM Data Challenge (since 2009) <http://icwsm.cs.umbc.edu/>_Kaggle Competition Data <https://www.kaggle.com/>_KDD Cup by Tencent 2012 <http://www.kddcup2012.org/>_Localytics Data Visualization Challenge <https://github.com/localytics/data-viz-challenge>_Netflix Prize <http://www.netflixprize.com/leaderboard>_Space Apps Challenge <https://2015.spaceappschallenge.org>_Telecom Italia Big Data Challenge <https://dandelion.eu/datamine/open-big-data/>_Yelp Dataset Challenge <http://www.yelp.com/dataset_challenge>_
Economics
American Economic Ass (AEA) <https://www.aeaweb.org/RFE/toc.php?show=complete>_EconData from UMD <http://inforumweb.umd.edu/econdata/econdata.html>_Economic Freedom of the World Data <http://www.freetheworld.com/datasets_efw.html>_Historical MacroEconomc Statistics <http://www.historicalstatistics.org/>_International Trade Statistics <http://www.econostatistics.co.za/>_Internet Product Code Database <http://www.upcdatabase.com/>_Joint External Debt Data Hub <http://www.jedh.org/>_Jon Haveman International Trade Data Links <http://www.macalester.edu/research/economics/PAGE/HAVEMAN/Trade.Resources/TradeData.html>_OpenCorporates Database of Companies in the World <https://opencorporates.com/>_Our World in Data <http://ourworldindata.org/>_SciencesPo World Trade Gravity Datasets <http://econ.sciences-po.fr/thierry-mayer/data>_The Atlas of Economic Complexity <http://atlas.cid.harvard.edu>_The Center for International Data <http://cid.econ.ucdavis.edu>_The Observatory of Economic Complexity <http://atlas.media.mit.edu/en/>_UN Commodity Trade Statistics <http://comtrade.un.org/db/>_UN Human Development Reports <http://hdr.undp.org/en>_
Energy
AMPds <http://ampds.org/>_BLUEd <http://nilm.cmubi.org/>_COMBED <http://combed.github.io/>_Dataport <https://dataport.pecanstreet.org/>_ECO <http://www.vs.inf.ethz.ch/res/show.html?what=eco-data>_EIA <http://www.eia.gov/electricity/data/eia923/>_HFED <http://hfed.github.io/>_iAWE <http://iawe.github.io/>_Plaid <http://plaidplug.com/>_REDD <http://redd.csail.mit.edu/>_UK-Dale <http://www.doc.ic.ac.uk/~dk3810/data/>_
Finance
CBOE Futures Exchange <http://cfe.cboe.com/Data/>_Google Finance <https://www.google.com/finance>_Google Trends <http://www.google.com/trends?q=google&ctab=0&geo=all&date=all&sort=0>_NASDAQ <https://data.nasdaq.com/>_OANDA <http://www.oanda.com/>_OSU Financial data <http://fisher.osu.edu/fin/fdf/osudata.htm>_Quandl <https://www.quandl.com/>_St Louis Federal <https://research.stlouisfed.org/fred2/>_Yahoo Finance <http://finance.yahoo.com/>_
Geology
Earth Models <http://www.earthmodels.org/>_Smithsonian Institution Global Volcano and Eruption Database <http://volcano.si.edu/>_USGS Earthquake Archives <http://earthquake.usgs.gov/earthquakes/search/>_
GeoSpace/GIS
BODC - marine data of ~22K vars <http://www.bodc.ac.uk/data/where_to_find_data/>_Cambridge, MA, US, GIS data on GitHub <http://cambridgegis.github.io/gisdata.html>_EOSDIS - NASA's earth observing system data <http://sedac.ciesin.columbia.edu/data/sets/browse>_Factual Global Location Data <https://www.factual.com/>_Geo Spatial Data from ASU <http://geodacenter.asu.edu/datalist/>_Geo Wiki Project - Citizen-driven Environmental Monitoring <http://geo-wiki.org/>_GeoNames Worldwide <http://www.geonames.org/>_Global Administrative Areas Database (GADM) <http://www.gadm.org/>_International Institute for Systems Analysis - GIS Datasets <http://www.iiasa.ac.at/web/home/research/modelsData/Models--Tools--Data.en.html>_Landsat 8 on AWS <https://aws.amazon.com/public-data-sets/landsat/>_List of all countries in all languages <https://github.com/umpirsky/country-list>_Natural Earth - vectors and rasters of the world <http://www.naturalearthdata.com/>_OpenAddresses <http://openaddresses.io/>_OpenStreetMap (OSM) <http://wiki.openstreetmap.org/wiki/Downloading_data>_Reverse Geocoder using OSM data <https://github.com/kno10/reversegeocode>_ &additional high-resolution data files <http://data.ub.uni-muenchen.de/61/>_TIGER/Line - U.S. boundaries and roads <http://www.census.gov/geo/maps-data/data/tiger-line.html>_TwoFishes - Foursquare's coarse geocoder <https://github.com/foursquare/twofishes>_TZ Timezones shapfiles <http://efele.net/maps/tz/world/>_UN Environmental Data <http://geodata.grid.unep.ch/>_World countries in multiple formats <https://github.com/mledoze/countries>_
Government
Alberta, Province of Canada <http://open.alberta.ca>_Antwerp, Belgium <http://opendata.antwerpen.be/datasets>_Argentina (non official) <http://datar.noip.me/>_Argentina <http://datos.argentina.gob.ar/>_Austin, TX, US <https://data.austintexas.gov/>_Australia (abs.gov.au) <http://www.abs.gov.au/AUSSTATS/[email protected]/DetailsPage/3301.02009?OpenDocument>_Australia (data.gov.au) <https://data.gov.au/>_Austria (data.gv.at) <https://www.data.gv.at/>_Baton Rouge, LA, US <https://data.brla.gov/>_Belgium <http://data.gov.be/nl/datasets>_Brazil <http://dados.gov.br/dataset>_Buenos Aires, Argentina <http://data.buenosaires.gob.ar/>_Calgary, AB, Canada <https://data.calgary.ca/OpenData/Pages/DatasetListingAlphabetical.aspx>_Cambridge, MA, US <https://data.cambridgema.gov/>_Canada <http://open.canada.ca/en?lang=En&n=5BCD274E-1>_Chicago <https://data.cityofchicago.org/>_Dallas Open Data <https://www.dallasopendata.com/>_DataBC - data from the Province of British Columbia <http://www.data.gov.bc.ca/>_Denver Open Data <http://data.denvergov.org//>_Durham, NC Open Data <https://opendurham.nc.gov/explore/>_Edmonton, AB, Canada <https://data.edmonton.ca/>_England LGInform <http://lginform.local.gov.uk/>_EuroStat <http://ec.europa.eu/eurostat/data/database>_FedStats <http://fedstats.sites.usa.gov/>_Finland <https://www.opendata.fi/en>_France <https://www.data.gouv.fr/en/datasets/>_Fredericton, NB, Canada <http://www.fredericton.ca/en/citygovernment/Catalogue.asp>_Gatineau, QC, Canada <http://www.gatineau.ca/donneesouvertes/default_fr.aspx>_Germany <https://www-genesis.destatis.de/genesis/online>_Ghent, Belgium <https://data.stad.gent/datasets>_Glasgow, Scotland, UK <https://data.glasgow.gov.uk/>_Guardian world governments <http://www.guardian.co.uk/world-government-data>_Halifax, NS, Canada <http://www.halifax.ca/opendata/index.php>_Helsinki Region, Finland <http://www.hri.fi/en/>_Houston Open Data <http://data.ohouston.org>_Indian Government Data <https://data.gov.in/>_Indonesian Data Portal <http://data.go.id/>_Laval, QC, Canada <http://www.laval.ca/Pages/Fr/Citoyens/donnees.aspx>_London Datastore, UK <http://data.london.gov.uk/dataset>_London, ON, Canada <http://www.london.ca/city-hall/open-data/Pages/default.aspx>_Los Angeles Open Data <https://data.lacity.org/>_MassGIS, Massachusetts, U.S. <http://www.mass.gov/anf/research-and-tech/it-serv-and-support/application-serv/office-of-geographic-information-massgis/>_Mexico <http://catalogo.datos.gob.mx/dataset>_Missisauga, ON, Canada <http://www.mississauga.ca/portal/residents/publicationsopendatacatalogue>_Moncton, NB, Canada <http://www.moncton.ca/Government/Terms_of_use/Open_Data_Purpose/Data_Catalogue.htm>_Montreal, QC, Canada <http://donnees.ville.montreal.qc.ca/>_Netherlands <https://data.overheid.nl/>_New Zealand <http://www.stats.govt.nz/browse_for_stats.aspx>_NYC betanyc <http://betanyc.us/>_NYC Open Data <https://nycplatform.socrata.com/>_OECD <http://www.oecd.org/document/0,3746,en_2649_201185_46462759_1_1_1_1,00.html>_Oklahoma <https://data.ok.gov/>_Open Government Data (OGD) Platform India <https://data.gov.in/>_Oregon <https://data.oregon.gov/>_Ottawa, ON, Canada <http://data.ottawa.ca/en/>_Portland, Oregon <https://www.portlandoregon.gov/28130>_Puerto Rico Government <https://data.pr.gov//>_Quebec City, QC, Canada <http://donnees.ville.quebec.qc.ca/>_Quebec Province of Canada <http://donnees.gouv.qc.ca/>_Regina SK, Canada <http://open.regina.ca/>_Rio de Janeiro, Brazil <http://data.rio.rj.gov.br/>_Romania <http://data.gov.ro/>_Russia <http://data.gov.ru>_San Francisco Data sets <http://datasf.org/>_Saskatchewan, Province of Canada <http://opendatask.ca/data/>_Seattle <https://data.seattle.gov/>_Singapore Government Data <https://data.gov.sg/>_South Africa <http://beta2.statssa.gov.za/>_South Africa Trade Statistics <http://www.econostatistics.co.za/>_State of Utah, US <https://opendata.utah.gov/>_Switzerland <http://www.opendata.admin.ch/>_Texas Open Data <https://data.texas.gov/>_The World Bank <http://wdronline.worldbank.org/>_Toronto, ON, Canada <http://www1.toronto.ca/wps/portal/contentonly?vgnextoid=1a66e03bb8d1e310VgnVCM10000071d60f89RCRD>_U.K. Government Data <http://data.gov.uk/data>_U.S. American Community Survey <http://www.census.gov/acs/www/data_documentation/data_release_info/>_U.S. CDC Public Health datasets <http://www.cdc.gov/nchs/data_access/ftp_data.htm>_U.S. Census Bureau <http://www.census.gov/data.html>_U.S. Department of Housing and Urban Development (HUD) <http://www.huduser.gov/portal/datasets/pdrdatas.html>_U.S. Federal Government Agencies <http://www.data.gov/metrics>_U.S. Federal Government Data Catalog <http://catalog.data.gov/dataset>_U.S. Food and Drug Administration (FDA) <https://open.fda.gov/index.html>_U.S. National Center for Education Statistics (NCES) <http://nces.ed.gov/>_U.S. Open Government <http://www.data.gov/open-gov/>_UK 2011 Census Open Atlas Project <http://www.alex-singleton.com/r/2013/02/05/2011-census-open-atlas-project/>_United Nations <http://data.un.org/>_Uruguay <https://catalogodatos.gub.uy/>_Vancouver, BC Open Data Catalog <http://data.vancouver.ca/datacatalogue/>_Victoria, BC, Canada <http://www.victoria.ca/EN/main/city/open-data-catalogue.html>_
Healthcare
EHDP Large Health Data Sets <http://www.ehdp.com/vitalnet/datasets.htm>_Gapminder World demographic databases <http://www.gapminder.org/data/>_Medicare Coverage Database (MCD), U.S. <https://www.cms.gov/medicare-coverage-database/>_Medicare Data Engine of medicare.gov Data <https://data.medicare.gov/>_Medicare Data File <http://go.cms.gov/19xxPN4>_MeSH, the vocabulary thesaurus used for indexing articles for PubMed <https://www.nlm.nih.gov/mesh/filelist.html>_Number of Ebola Cases and Deaths in Affected Countries (2014) <https://data.hdx.rwlabs.org/dataset/ebola-cases-2014>_Open-ODS (structure of the UK NHS) <http://www.openods.co.uk>_The Cancer Genome Atlas project (TCGA) <https://tcga-data.nci.nih.gov/tcga/tcgaDownload.jsp>_ andBigQuery table <http://google-genomics.readthedocs.org/en/latest/use_cases/discover_public_data/isb_cgc_data.html>_World Health Organization Global Health Observatory <http://www.who.int/gho/en/>_
Image Processing
10k US Adult Faces Database <http://wilmabainbridge.com/facememorability2.html>_2GB of Photos of Cats <http://137.189.35.203/WebUI/CatDatabase/catData.html>_ orArchive version <https://web.archive.org/web/20150520175645/http://137.189.35.203/WebUI/CatDatabase/catData.html>_Affective Image Classification <http://www.imageemotion.org/>_Animals with attributes <http://attributes.kyb.tuebingen.mpg.de/>_Face Recognition Benchmark <http://www.face-rec.org/databases/>_ImageNet (in WordNet hierarchy) <http://www.image-net.org/>_Indoor Scene Recognition <http://web.mit.edu/torralba/www/indoor.html>_International Affective Picture System, UFL <http://csea.phhp.ufl.edu/media/iapsmessage.html>_Massive Visual Memory Stimuli, MIT <http://cvcl.mit.edu/MM/stimuli.html>_Several Shape-from-Silhouette Datasets <http://kaiwolf.no-ip.org/3d-model-repository.html>_Stanford Dogs Dataset <http://vision.stanford.edu/aditya86/ImageNetDogs/>_SUN database, MIT <http://groups.csail.mit.edu/vision/SUN/hierarchy.html>_The Oxford-IIIT Pet Dataset <http://www.robots.ox.ac.uk/~vgg/data/pets/>_YouTube Faces Database <http://www.cs.tau.ac.il/~wolf/ytfaces/>_
Machine Learning
Delve Datasets for classification and regression (Univ. of Toronto) <http://www.cs.toronto.edu/~delve/data/datasets.html>_Discogs Monthly Data <http://data.discogs.com/>_eBay Online Auctions (2012) <http://www.modelingonlineauctions.com/datasets>_IMDb Database <http://www.imdb.com/interfaces>_Keel Repository for classification, regression and time series <http://sci2s.ugr.es/keel/datasets.php>_Lending Club Loan Data <https://www.lendingclub.com/info/download-data.action>_Machine Learning Data Set Repository <http://mldata.org/>_Million Song Dataset <http://labrosa.ee.columbia.edu/millionsong/>_More Song Datasets <http://labrosa.ee.columbia.edu/millionsong/pages/additional-datasets>_MovieLens Data Sets <http://grouplens.org/datasets/movielens/>_RDataMining - "R and Data Mining" ebook data <http://www.rdatamining.com/data>_Registered Meteorites on Earth <http://healthintelligence.drupalgardens.com/content/registered-meteorites-has-impacted-earth-visualized>_Restaurants Health Score Data in San Francisco <http://missionlocal.org/san-francisco-restaurant-health-inspections/>_UCI Machine Learning Repository <http://archive.ics.uci.edu/ml/>_Yahoo! Ratings and Classification Data <http://webscope.sandbox.yahoo.com/catalog.php?datatype=r>_Labeled Faces in the Wild (LFW) <http://vis-www.cs.umass.edu/lfw/>_
Museums
Canada Science and Technology Museums Corporation's Open Data <http://techno-science.ca/en/data.php>_Cooper-Hewitt's Collection Database <https://github.com/cooperhewitt/collection>_Minneapolis Institute of Arts metadata <https://github.com/artsmia/collection>_Natural History Museum (London) Data Portal <http://data.nhm.ac.uk/>_Rijksmuseum Historical Art Collection <https://www.rijksmuseum.nl/en/api>_Tate Collection metadata <https://github.com/tategallery/collection>_The Getty vocabularies <http://vocab.getty.edu>_
Natural Language
Blogger Corpus <http://u.cs.biu.ac.il/~koppel/BlogCorpus.htm>_ClueWeb09 FACC <http://lemurproject.org/clueweb09/FACC1/>_ClueWeb12 FACC <http://lemurproject.org/clueweb12/FACC1/>_DBpedia - 4.58M things with 583M facts <http://wiki.dbpedia.org/Datasets>_Flickr Personal Taxonomies <http://www.isi.edu/~lerman/downloads/flickr/flickr_taxonomies.html>_Freebase.com of people, places, and things <http://www.freebase.com/>_Google Books Ngrams (2.2TB) <https://aws.amazon.com/datasets/google-books-ngrams/>_Google Web 5gram (1TB, 2006) <https://catalog.ldc.upenn.edu/LDC2006T13>_Gutenberg eBooks List <http://www.gutenberg.org/wiki/Gutenberg:Offline_Catalogs>_Hansards text chunks of Canadian Parliament <http://www.isi.edu/natural-language/download/hansard/>_Machine Comprehension Test (MCTest) of text from Microsoft Research <http://research.microsoft.com/en-us/um/redmond/projects/mctest/index.html>_Machine Translation of European languages <http://statmt.org/wmt11/translation-task.html#download>_SaudiNewsNet Collection of Saudi Newspaper Articles (Arabic, 30K articles) <https://github.com/ParallelMazen/SaudiNewsNet>_SMS Spam Collection in English <http://www.dt.fee.unicamp.br/~tiago/smsspamcollection/>_USENET postings corpus of 2005~2011 <http://www.psych.ualberta.ca/~westburylab/downloads/usenetcorpus.download.html>_Wikidata - Wikipedia databases <https://www.wikidata.org/wiki/Wikidata:Database_download>_Wikipedia Links data - 40 Million Entities in Context <https://code.google.com/p/wiki-links/downloads/list>_WordNet databases and tools <http://wordnet.princeton.edu/wordnet/download/>_
Physics
CERN Open Data Portal <http://opendata.cern.ch/>_NASA Exoplanet Archive <http://exoplanetarchive.ipac.caltech.edu/>_NSSDC (NASA) data of 550 space spacecraft <http://nssdc.gsfc.nasa.gov/nssdc/obtaining_data.html>_Sloan Digital Sky Survey (SDSS) - Mapping the Universe <http://www.sdss.org/>_
Psychology/Cognition
OSU Cognitive Modeling Repository Datasets <http://www.cmr.osu.edu/browse/datasets>_
Public Domains
Amazon <http://aws.amazon.com/datasets/>_Archive.org Datasets <https://archive.org/details/datasets>_CMU JASA data archive <http://lib.stat.cmu.edu/jasadata/>_CMU StatLab collections <http://lib.stat.cmu.edu/datasets/>_Data360 <http://www.data360.org/index.aspx>_Datamob.org <http://datamob.org/datasets>_Google <http://www.google.com/publicdata/directory>_Infochimps <http://www.infochimps.com/>_KDNuggets Data Collections <http://www.kdnuggets.com/datasets/index.html>_Microsoft Azure Data Market Free DataSets <http://datamarket.azure.com/browse/data?price=free>_Numbray <http://numbrary.com/>_Reddit Datasets <https://www.reddit.com/r/datasets>_RevolutionAnalytics Collection <http://packages.revolutionanalytics.com/datasets/>_Sample R data sets <http://stat.ethz.ch/R-manual/R-patched/library/datasets/html/00Index.html>_Stats4Stem R data sets <http://www.stats4stem.org/data-sets.html>_StatSci.org <http://www.statsci.org/datasets.html>_The Washington Post List <http://www.washingtonpost.com/wp-srv/metro/data/datapost.html>_UCLA SOCR data collection <http://wiki.stat.ucla.edu/socr/index.php/SOCR_Data>_UFO Reports <http://www.nuforc.org/webreports.html>_Wikileaks 911 pager intercepts <https://911.wikileaks.org/files/index.html>_Yahoo Webscope <http://webscope.sandbox.yahoo.com/catalog.php>_
Search Engines
Academic Torrents of data sharing from UMB <http://academictorrents.com/>_Archive-it from Internet Archive <https://www.archive-it.org/explore?show=Collections>_Datahub.io <https://datahub.io/dataset>_DataMarket (Qlik) <https://datamarket.com/data/list/?q=all>_Harvard Dataverse Network of scientific data <https://dataverse.harvard.edu/>_ICPSR (UMICH) <http://www.icpsr.umich.edu/icpsrweb/ICPSR/index.jsp>_Institute of Education Sciences <http://eric.ed.gov>_National Technical Reports Library <https://ntrl.ntis.gov/NTRL/login.xhtml>_Open Data Certificates (beta) <https://certificates.theodi.org/en/datasets>_OpenDataNetwork - A search engine of all Socrata powered data portals <http://www.opendatanetwork.com/>_Statista.com - statistics and Studies <http://www.statista.com/>_Zenodo - An open dependable home for the long-tail of science <https://zenodo.org/collection/datasets>_
Social Networks
72 hours #gamergate Twitter Scrape <http://waxy.org/random/misc/gamergate_tweets.csv>_Ancestry.com Forum Dataset over 10 years <http://www.cs.cmu.edu/~jelsas/data/ancestry.com/>_Cheng-Caverlee-Lee September 2009 - January 2010 Twitter Scrape <https://archive.org/details/twitter_cikm_2010>_CMU Enron Email of 150 users <http://www.cs.cmu.edu/~enron/>_EDRM Enron EMail of 151 users, hosted on S3 <https://aws.amazon.com/datasets/enron-email-data/>_Facebook Data Scrape (2005) <https://archive.org/details/oxford-2005-facebook-matrix>_Facebook Social Networks from LAW (since 2007) <http://law.di.unimi.it/datasets.php>_Foursquare from UMN/Sarwat (2013) <https://archive.org/details/201309_foursquare_dataset_umn>_GetGlue - users rating TV shows <http://getglue-data.s3.amazonaws.com/getglue_sample.tar.gz>_GitHub Collaboration Archive <https://www.githubarchive.org/>_Google Scholar citation relations <http://www3.cs.stonybrook.edu/~leman/data/gscholar.db>_Mobile Social Networks from UMASS <https://kdl.cs.umass.edu/display/public/Mobile+Social+Networks>_Network Twitter Data <http://snap.stanford.edu/data/higgs-twitter.html>_Reddit Comments <https://www.reddit.com/r/datasets/comments/3bxlg7/i_have_every_publicly_available_reddit_comment/>_Skytrax' Air Travel Reviews Dataset <https://github.com/quankiquanki/skytrax-reviews-dataset>_Social Twitter Data <http://snap.stanford.edu/data/egonets-Twitter.html>_SourceForge.net Research Data <http://www3.nd.edu/~oss/Data/data.html>_Twitter Data for Sentiment Analysis <http://help.sentiment140.com/for-students/>_Twitter Graph of entire Twitter site <http://an.kaist.ac.kr/traces/WWW2010.html>_Twitter Scrape Calufa May 2011 <http://archive.org/details/2011-05-calufa-twitter-sql>_UNIMI/LAW Social Network Datasets <http://law.di.unimi.it/datasets.php>_Yahoo! Graph and Social Data <http://webscope.sandbox.yahoo.com/catalog.php?datatype=g>_Youtube Video Social Graph in 2007,2008 <http://netsg.cs.sfu.ca/youtubedata/>_
Social Sciences
Canadian Legal Information Institute <https://www.canlii.org/en/index.php>_Center for Systemic Peace Datasets - Conflict Trends, Polities, State Fragility, etc <http://www.systemicpeace.org/>_Correlates of War Project <http://www.correlatesofwar.org/>_Cryptome Conspiracy Theory Items <http://cryptome.org>_Datacards <http://datacards.org>_European Social Survey <http://www.europeansocialsurvey.org/data/>_FBI Hate Crime 2013 - aggregated data <https://github.com/emorisse/FBI-Hate-Crime-Statistics/tree/master/2013>_GDELT Global Events Database <http://gdeltproject.org/data.html>_General Social Survey (GSS) since 1972 <http://gss.norc.org>_German Social Survey <http://www.gesis.org/en/home/>_Global Religious Futures Project <http://www.globalreligiousfutures.org/>_Institute for Demographic Studies <http://www.ined.fr/en/>_International Networks Archive <http://www.princeton.edu/~ina/>_International Social Survey Program ISSP <http://www.issp.org>_International Studies Compendium Project <http://www.isacompendium.com/public/>_James McGuire Cross National Data <http://jmcguire.faculty.wesleyan.edu/welcome/cross-national-data/>_MIT Reality Mining Dataset <http://realitycommons.media.mit.edu/realitymining.html>_Paul Hensel General International Data Page <http://www.paulhensel.org/dataintl.html>_PewResearch Internet Survey Project <http://www.pewinternet.org/datasets/pages/2/>_PewResearch Society Data Collection <http://www.pewresearch.org/data/download-datasets/>_Political Polarity Data <http://www3.cs.stonybrook.edu/~leman/data/14-icwsm-political-polarity-data.zip>_StackExchange Data Explorer <http://data.stackexchange.com/help>_Terrorism Research and Analysis Consortium <http://www.trackingterrorism.org/>_Texas Inmates Executed Since 1984 <http://www.tdcj.state.tx.us/death_row/dr_executed_offenders.html>_The MacroData Guide by Norsk samfunnsvitenskapelig datatjeneste <http://nsd.uib.no>_Titanic Survival Data Set <https://github.com/caesar0301/awesome-public-datasets/tree/master/Datasets>_UCB's Archive of Social Science Data (D-Lab) <http://ucdata.berkeley.edu/>_UCLA Social Sciences Data Archive <http://dataarchives.ss.ucla.edu/Home.DataPortals.htm>_UN Civil Society Database <http://esango.un.org/civilsociety/>_Universities Worldwide <http://univ.cc/>_UPJOHN for Labor Employment Research <http://www.upjohn.org/services/resources/employment-research-data-center>_
Sports
Betfair Historical Exchange Data <http://data.betfair.com/>_Cricsheet Matches (cricket) <http://cricsheet.org/>_Ergast Formula 1, from 1950 up to date (API) <http://ergast.com/mrd/db>_Football/Soccer resources (data and APIs) <http://www.jokecamp.com/blog/guide-to-football-and-soccer-data-and-apis/>_Lahman's Baseball Database <http://www.seanlahman.com/baseball-archive/statistics/>_Retrosheet Baseball Statistics <http://www.retrosheet.org/game.htm>_
Time Series
Databanks International Cross National Time Series Data Archive <http://www.cntsdata.com>_Hard Drive Failure Rates <https://www.backblaze.com/hard-drive-test-data.html>_Heart Rate Time Series from MIT <http://ecg.mit.edu/time-series/>_Time Series Data Library (TSDL) from MU <https://datamarket.com/data/list/?q=provider:tsdl>_UC Riverside Time Series Dataset <http://www.cs.ucr.edu/~eamonn/time_series_data/>_
Transportation
Airlines OD Data 1987-2008 <http://stat-computing.org/dataexpo/2009/the-data.html>_Bay Area Bike Share Data <http://www.bayareabikeshare.com/open-data>_Bike Share Systems (BSS) collection <https://github.com/BetaNYC/Bike-Share-Data-Best-Practices/wiki/Bike-Share-Data-Systems>_GeoLife GPS Trajectory from Microsoft Research <http://research.microsoft.com/en-us/downloads/b16d359d-d164-469e-9fd4-daa38f2b2e13/>_German train system by Deutsche Bahn <http://data.deutschebahn.com/datasets/>_Hubway Million Rides in MA <http://hubwaydatachallenge.org/trip-history-data/>_Marine Traffic - ship tracks, port calls and more <http://www.marinetraffic.com/de/ais-api-services>_Montreal BIXI Bike Share <https://montreal.bixi.com/donn%C3%A9es-libre-service>_NYC Taxi Trip Data 2009- <http://www.nyc.gov/html/tlc/html/about/trip_record_data.shtml>_NYC Taxi Trip Data 2013 (FOIA/FOILed) <https://archive.org/details/nycTaxiTripData2013>_NYC Uber trip data April 2014 to September 2014 <https://github.com/fivethirtyeight/uber-tlc-foil-response>_OpenFlights - airport, airline and route data <http://openflights.org/data.html>_Plane Crash Database, since 1920 <http://www.planecrashinfo.com/database.htm>_RITA Airline On-Time Performance data <http://www.transtats.bts.gov/Tables.asp?DB_ID=120>_RITA/BTS transport data collection (TranStat) <http://www.transtats.bts.gov/DataIndex.asp>_Toronto Bike Share Stations (XML file) <http://www.bikesharetoronto.com/data/stations/bikeStations.xml>_Transport for London (TFL) <https://tfl.gov.uk/info-for/open-data-users/our-feeds>_Travel Tracker Survey (TTS) for Chicago <http://www.cmap.illinois.gov/data/transportation/travel-tracker-survey>_U.S. Bureau of Transportation Statistics (BTS) <http://www.rita.dot.gov/bts/>_U.S. Domestic Flights 1990 to 2009 <http://academictorrents.com/details/a2ccf94bbb4af222bf8e69dad60a68a29f310d9a>_U.S. Freight Analysis Framework since 2007 <http://ops.fhwa.dot.gov/freight/freight_analysis/faf/index.htm>_
Complementary Collections
Database of Scientific Code Contributions <https://mozillascience.org/collaborate>_- DataWrangling:
Some Datasets Available on the Web <http://www.datawrangling.com/some-datasets-available-on-the-web>_ - Inside-r:
Finding Data on the Internet <http://www.inside-r.org/howto/finding-data-internet>_ - OpenDataMonitor:
An overview of available open data resources in Europe <http://opendatamonitor.eu>_ - Quora:
Where can I find large datasets open to the public? <http://www.quora.com/Where-can-I-find-large-datasets-open-to-the-public>_ - RS.io:
100+ Interesting Data Sets for Statistics <http://rs.io/100-interesting-data-sets-for-statistics/>_ - StaTrek:
Leveraging open data to understand urban lives <http://xiaming.me/posts/2014/10/23/leveraging-open-data-to-understand-urban-lives/>_
Related repositories
codecrafters-io/build-your-own-x
Master programming by recreating your favorite technologies from scratch.
★ 536.9k
sindresorhus/awesome
😎 Awesome lists about all kinds of interesting topics
★ 490.8k
public-apis/public-apis
Public APIs is a community-curated GitHub repository listing free, publicly accessible APIs across a wide range of categories, with auth type, HTTPS, and CORS noted for each entry. It's a browsable reference, not a library to install.
★ 454.8k
freeCodeCamp/freeCodeCamp
freeCodeCamp.org's open-source codebase and curriculum. Learn math, programming, and computer science for free.
★ 453.4k
FREQUENTLY ASKED
Quick answers
How many stars does josephmisiti/awesome-public-datasets have?
josephmisiti/awesome-public-datasets has 13 GitHub stars — refresh the page for the live number, or check github.com/josephmisiti/awesome-public-datasets. TopGit mirrors GitHub's count but does not claim minute-by-minute accuracy.
Is josephmisiti/awesome-public-datasets open source?
Yes — josephmisiti/awesome-public-datasets ships under the MIT license, which makes its source code freely readable (and, depending on license terms, forkable and reusable). Source: github.com/josephmisiti/awesome-public-datasets.
What is josephmisiti/awesome-public-datasets?
josephmisiti/awesome-public-datasets (josephmisiti/awesome-public-datasets) is a multi-language project on GitHub. From the project's own README: An awesome list of high-quality open datasets in public domains (on-going).
Where can I see josephmisiti/awesome-public-datasets in action?
The project maintains a homepage at https://goo.gl/WZ8XAJ. The README tab on this page also usually contains screenshots and a quickstart.
Where do I read more about josephmisiti/awesome-public-datasets?
This TopGit page is a snapshot — the READ ME tab shows the project's own README content (links stripped, images preserved). The GitHub repository at github.com/josephmisiti/awesome-public-datasets is the definitive source.
Read full README in the tab above.