Downloads
In addition to the HTTP Archive source code being open source,
all of the data gathered is available as downloadable data dumps.
These dumps are also available in Google BigQuery thanks to Ilya Grigorik .
For more information see Ilya's blog post HTTP Archive + BigQuery = Web Performance Answers
and video .
And checkout all the custom queries shared on discuss.httparchive.org .
Data Dumps
The results from each crawl are saved as dump files in CSV format.
Dumps are made for both the desktop and mobile crawls.
The dump files do not contain the commands to create the database and tables.
To restore a dump:
Import the schema to create the tables.
cat httparchive_schema.sql | mysql -u MYSQL_USERNAME -p -h MYSQL_HOSTNAME MYSQL_DB
Import the desired dump file.
For CSV-formatted dumps, gunzip the file and rename it to either pages.csv
or requests.csv
, then run this command:
mysqlimport --local --fields-optionally-enclosed-by="\"" --fields-terminated-by=, --user=MYSQL_USERNAME -p MYSQL_DB MYSQLDUMP_FILE.csv
If you want to run a private instance of the source code, you need to also import the stats and crawls dumps.
Files
These files define the schema and the meta-level tables:
schema - the schema for the tables referenced in the data dumps
stats - the aggregated stats for all crawls
urls - the URLs used in the most recent crawl
crawls - meta-information about all of the crawls
There's a download file for each type of crawl:
Apr 1 2022: Chrome: pages , requests Android: pages , requests
Mar 1 2022: Chrome: pages , requests Android: pages , requests
Feb 1 2022: Chrome: pages , requests Android: pages , requests
Jan 1 2022: Chrome: pages , requests Android: pages , requests
Dec 1 2021: Chrome: pages , requests Android: pages , requests
Nov 1 2021: Chrome: pages , requests Android: pages , requests
Oct 1 2021: Chrome: pages , requests Android: pages , requests
Sep 1 2021: Chrome: pages , requests Android: pages , requests
Aug 1 2021: Chrome: pages , requests Android: pages , requests
Jul 1 2021: Chrome: pages , requests Android: pages , requests
Jun 1 2021: Chrome: pages , requests Android: pages , requests
May 1 2021: Chrome: pages , requests Android: pages , requests
Apr 1 2021: Chrome: pages , requests Android: pages , requests
Mar 1 2021: Chrome: pages , requests Android: pages , requests
Feb 1 2021: Chrome: pages , requests Android: pages , requests
Jan 1 2021: Chrome: pages , requests Android: pages , requests
Dec 1 2020: Chrome: pages , requests Android: pages , requests
Nov 1 2020: Chrome: pages , requests Android: pages , requests
Oct 1 2020: Chrome: pages , requests Android: pages , requests
Sep 1 2020: Chrome: pages , requests Android: pages , requests
Aug 1 2020: Chrome: pages , requests Android: pages , requests
Jul 1 2020: Chrome: pages , requests Android: pages , requests
Jun 1 2020: Chrome: pages , requests Android: pages , requests
May 1 2020: Chrome: pages , requests Android: pages , requests
Apr 1 2020: Chrome: pages , requests Android: pages , requests
Mar 1 2020: Chrome: pages , requests Android: pages , requests
Feb 1 2020: Chrome: pages , requests Android: pages , requests
Jan 1 2020: Chrome: pages , requests Android: pages , requests
Dec 5 2019: Chrome: pages , requests
Nov 1 2019: Chrome: pages , requests Android: pages , requests
Oct 1 2019: Chrome: pages , requests Android: pages , requests
Sep 1 2019: Chrome: pages , requests Android: pages , requests
Aug 1 2019: Chrome: pages , requests Android: pages , requests
Jul 1 2019: Chrome: pages , requests Android: pages , requests
Jun 1 2019: Chrome: pages , requests Android: pages , requests
May 1 2019: Chrome: pages , requests Android: pages , requests
Apr 1 2019: Chrome: pages , requests Android: pages , requests
Mar 1 2019: Chrome: pages , requests Android: pages , requests
Feb 1 2019: Chrome: pages , requests Android: pages , requests
Jan 1 2019: Chrome: pages , requests Android: pages , requests
Dec 15 2018: Chrome: pages , requests Android: pages , requests
Dec 1 2018: Chrome: pages , requests Android: pages , requests
Nov 15 2018: Chrome: pages , requests Android: pages , requests
Nov 1 2018: Chrome: pages , requests Android: pages , requests
Oct 15 2018: Chrome: pages , requests Android: pages , requests
Oct 1 2018: Chrome: pages , requests Android: pages , requests
Sep 15 2018: Chrome: pages , requests Android: pages , requests
Sep 1 2018: Chrome: pages , requests Android: pages , requests
Aug 15 2018: Chrome: pages , requests Android: pages , requests
Aug 1 2018: Chrome: pages , requests Android: pages , requests
Jul 15 2018: Chrome: pages , requests Android: pages , requests
Jul 1 2018: Chrome: pages , requests Android: pages , requests
Jun 15 2018: Chrome: pages , requests Android: pages , requests
Jun 1 2018: Chrome: pages , requests Android: pages , requests
May 15 2018: Chrome: pages , requests Android: pages , requests
May 1 2018: Chrome: pages , requests Android: pages , requests
Apr 15 2018: Chrome: pages , requests Android: pages , requests
Apr 1 2018: Chrome: pages , requests Android: pages , requests
Mar 15 2018: Chrome: pages , requests Android: pages , requests
Mar 1 2018: Chrome: pages , requests Android: pages , requests
Feb 15 2018: Chrome: pages , requests Android: pages , requests
Feb 1 2018: Chrome: pages , requests Android: pages , requests
Jan 15 2018: Chrome: pages , requests Android: pages , requests
Jan 1 2018: Chrome: pages , requests Android: pages , requests
Dec 15 2017: Chrome: pages , requests Android: pages , requests
Dec 1 2017: Chrome: pages , requests Android: pages , requests
Nov 15 2017: Chrome: pages , requests Android: pages , requests
Nov 1 2017: Chrome: pages , requests Android: pages , requests
Oct 16 2017: Chrome: pages , requests Android: pages , requests
Oct 1 2017: Chrome: pages , requests Android: pages , requests
Sep 15 2017: Chrome: pages , requests Android: pages , requests
Sep 1 2017: Chrome: pages , requests Android: pages , requests
Aug 15 2017: Chrome: pages , requests Android: pages , requests
Aug 1 2017: Chrome: pages , requests Android: pages , requests
Jul 15 2017: Chrome: pages , requests Android: pages , requests
Jul 1 2017: Chrome: pages , requests Android: pages , requests
Jun 15 2017: Chrome: pages , requests Android: pages , requests
Jun 1 2017: Chrome: pages , requests Android: pages , requests
May 15 2017: Chrome: pages , requests Android: pages , requests
May 1 2017: Chrome: pages , requests Android: pages , requests
Apr 15 2017: Chrome: pages , requests Android: pages , requests
Apr 1 2017: Chrome: pages , requests Android: pages , requests
Mar 15 2017: Chrome: pages , requests Android: pages , requests
Mar 1 2017: Chrome: pages , requests Android: pages , requests
Feb 15 2017: Chrome: pages , requests Android: pages , requests
Feb 1 2017: Chrome: pages , requests Android: pages , requests
Jan 15 2017: Chrome: pages , requests Android: pages , requests
Jan 1 2017: Chrome: pages , requests Android: pages , requests
Dec 15 2016: Chrome: pages , requests Android: pages , requests
Dec 2 2016: Chrome: pages , requests Android: pages , requests
Nov 15 2016: Chrome: pages , requests Android: pages , requests
Nov 1 2016: Chrome: pages , requests Android: pages , requests
Oct 15 2016: Chrome: pages , requests Android: pages , requests
Oct 1 2016: Chrome: pages , requests Android: pages , requests
Sep 15 2016: Chrome: pages , requests Android: pages , requests
Sep 1 2016: Chrome: pages , requests Android: pages , requests
Aug 15 2016: Chrome: pages , requests Android: pages , requests
Aug 1 2016: Chrome: pages , requests Android: pages , requests
Jul 15 2016: Chrome: pages , requests Android: pages , requests
Jul 1 2016: Chrome: pages , requests Android: pages , requests
Jun 15 2016: Chrome: pages , requests Android: pages , requests IE: pages , requests iPhone: pages , requests
Jun 1 2016: Chrome: pages , requests Android: pages , requests IE: pages , requests iPhone: pages , requests
May 15 2016: Chrome: pages , requests Android: pages , requests IE: pages , requests iPhone: pages , requests
May 1 2016: Chrome: pages , requests Android: pages , requests IE: pages , requests iPhone: pages , requests
Apr 15 2016: Chrome: pages , requests Android: pages , requests IE: pages , requests iPhone: pages , requests
Apr 1 2016: Chrome: pages , requests Android: pages , requests IE: pages , requests iPhone: pages , requests
Mar 15 2016: Chrome: pages , requests Android: pages , requests IE: pages , requests iPhone: pages , requests
Mar 1 2016: Chrome: pages , requests Android: pages , requests IE: pages , requests iPhone: pages , requests
Feb 15 2016: Chrome: pages , requests Android: pages , requests IE: pages , requests iPhone: pages , requests
Feb 1 2016: Chrome: pages , requests Android: pages , requests IE: pages , requests iPhone: pages , requests
Jan 15 2016: Chrome: pages , requests Android: pages , requests IE: pages , requests iPhone: pages , requests
Jan 1 2016: Chrome: pages , requests Android: pages , requests IE: pages , requests iPhone: pages , requests
Dec 15 2015: Chrome: pages , requests Android: pages , requests IE: pages , requests iPhone: pages , requests
Dec 1 2015: Chrome: pages , requests Android: pages , requests IE: pages , requests iPhone: pages , requests
Nov 15 2015: Chrome: pages , requests Android: pages , requests IE: pages , requests iPhone: pages , requests
Nov 1 2015: Chrome: pages , requests Android: pages , requests IE: pages , requests iPhone: pages , requests
Oct 15 2015: Chrome: pages , requests Android: pages , requests IE: pages , requests iPhone: pages , requests
Oct 1 2015: IE: pages , requests iPhone: pages , requests
Sep 15 2015: IE: pages , requests iPhone: pages , requests
Sep 1 2015: IE: pages , requests iPhone: pages , requests
Aug 15 2015: IE: pages , requests iPhone: pages , requests
Aug 1 2015: IE: pages , requests iPhone: pages , requests
Jul 15 2015: IE: pages , requests iPhone: pages , requests
Jul 1 2015: IE: pages , requests iPhone: pages , requests
Jun 15 2015: IE: pages , requests iPhone: pages , requests
Jun 1 2015: IE: pages , requests iPhone: pages , requests
May 15 2015: IE: pages , requests iPhone: pages , requests
May 1 2015: IE: pages , requests iPhone: pages , requests
Apr 15 2015: IE: pages , requests iPhone: pages , requests
Apr 1 2015: IE: pages , requests iPhone: pages , requests
Mar 15 2015: IE: pages , requests iPhone: pages , requests
Mar 1 2015: IE: pages , requests iPhone: pages , requests
Feb 15 2015: IE: pages , requests iPhone: pages , requests
Feb 1 2015: IE: pages , requests iPhone: pages , requests
Jan 15 2015: IE: pages , requests iPhone: pages , requests
Jan 1 2015: IE: pages , requests iPhone: pages , requests
Dec 15 2014: IE: pages , requests iPhone: pages , requests
Dec 1 2014: IE: pages , requests iPhone: pages , requests
Nov 15 2014: IE: pages , requests iPhone: pages , requests
Nov 1 2014: IE: pages , requests iPhone: pages , requests
Oct 15 2014: IE: pages , requests iPhone: pages , requests
Oct 1 2014: IE: pages , requests iPhone: pages , requests
Sep 15 2014: IE: pages , requests iPhone: pages , requests
Sep 1 2014: IE: pages , requests iPhone: pages , requests
Aug 15 2014: IE: pages , requests iPhone: pages , requests
Aug 1 2014: IE: pages , requests iPhone: pages , requests
Jul 15 2014: IE: pages , requests iPhone: pages , requests
Jul 1 2014: IE: pages , requests iPhone: pages , requests
Jun 15 2014: IE: pages , requests
Jun 1 2014: IE: pages , requests iPhone: pages , requests
May 15 2014: IE: pages , requests iPhone: pages , requests
May 1 2014: IE: pages , requests iPhone: pages , requests
Apr 15 2014: IE: pages , requests iPhone: pages , requests
Apr 1 2014: IE: pages , requests iPhone: pages , requests
Mar 15 2014: IE: pages , requests iPhone: pages , requests
Mar 1 2014: IE: pages , requests iPhone: pages , requests
Feb 15 2014: IE: pages , requests iPhone: pages , requests
Feb 1 2014: IE: pages , requests iPhone: pages , requests
Jan 15 2014: IE: pages , requests iPhone: pages , requests
Jan 1 2014: IE: pages , requests iPhone: pages , requests
Dec 15 2013: IE: pages , requests iPhone: pages , requests
Nov 15 2013: IE: pages , requests iPhone: pages , requests
Nov 1 2013: IE: pages , requests iPhone: pages , requests
Oct 15 2013: IE: pages , requests iPhone: pages , requests
Oct 1 2013: IE: pages , requests iPhone: pages , requests
Sep 15 2013: IE: pages , requests iPhone: pages , requests
Sep 1 2013: IE: pages , requests iPhone: pages , requests
Aug 15 2013: IE: pages , requests iPhone: pages , requests
Aug 1 2013: IE: pages , requests iPhone: pages , requests
Jul 15 2013: IE: pages , requests iPhone: pages , requests
Jul 1 2013: IE: pages , requests iPhone: pages , requests
Jun 15 2013: IE: pages , requests iPhone: pages , requests
Jun 1 2013: IE: pages , requests iPhone: pages , requests
May 15 2013: IE: pages , requests iPhone: pages , requests
May 1 2013: IE: pages , requests iPhone: pages , requests
Apr 15 2013: IE: pages , requests iPhone: pages , requests
Apr 1 2013: IE: pages , requests iPhone: pages , requests
Mar 15 2013: IE: pages , requests iPhone: pages , requests
Mar 1 2013: IE: pages , requests iPhone: pages , requests
Feb 15 2013: IE: pages , requests iPhone: pages , requests
Feb 1 2013: IE: pages , requests iPhone: pages , requests
Jan 15 2013: IE: pages , requests iPhone: pages , requests
Jan 1 2013: IE: pages , requests iPhone: pages , requests
Dec 15 2012: IE: pages , requests iPhone: pages , requests
Dec 1 2012: IE: pages , requests iPhone: pages , requests
Nov 15 2012: IE: pages , requests iPhone: pages , requests
Nov 1 2012: IE: pages , requests iPhone: pages , requests
Oct 15 2012: IE: pages , requests iPhone: pages , requests
Oct 1 2012: IE: pages , requests iPhone: pages , requests
Sep 15 2012: IE: pages , requests iPhone: pages , requests
Sep 1 2012: IE: pages , requests iPhone: pages , requests
Aug 15 2012: IE: pages , requests iPhone: pages , requests
Aug 1 2012: IE: pages , requests iPhone: pages , requests
Jul 15 2012: IE: pages , requests iPhone: pages , requests
Jul 1 2012: IE: pages , requests iPhone: pages , requests
Jun 15 2012: IE: pages , requests iPhone: pages , requests
Jun 1 2012: IE: pages , requests iPhone: pages , requests
May 15 2012: IE: pages , requests iPhone: pages , requests
May 1 2012: IE: pages , requests iPhone: pages , requests
Apr 15 2012: IE: pages , requests iPhone: pages , requests
Apr 1 2012: IE: pages , requests iPhone: pages , requests
Mar 15 2012: IE: pages , requests iPhone: pages , requests
Mar 1 2012: IE: pages , requests iPhone: pages , requests
Feb 15 2012: IE: pages , requests iPhone: pages , requests
Feb 1 2012: IE: pages , requests iPhone: pages , requests
Jan 15 2012: IE: pages , requests iPhone: pages , requests
Jan 1 2012: IE: pages , requests iPhone: pages , requests
Dec 15 2011: IE: pages , requests iPhone: pages , requests
Dec 1 2011: IE: pages , requests iPhone: pages , requests
Nov 15 2011: IE: pages , requests iPhone: pages , requests
Nov 1 2011: IE: pages , requests iPhone: pages , requests
Oct 15 2011: IE: pages , requests iPhone: pages , requests
Oct 1 2011: IE: pages , requests iPhone: pages , requests
Sep 15 2011: IE: pages , requests iPhone: pages , requests
Sep 1 2011: IE: pages , requests iPhone: pages , requests
Aug 15 2011: IE: pages , requests iPhone: pages , requests
Aug 1 2011: IE: pages , requests iPhone: pages , requests
Jul 15 2011: IE: pages , requests iPhone: pages , requests
Jul 1 2011: IE: pages , requests iPhone: pages , requests
Jun 15 2011: IE: pages , requests iPhone: pages , requests
Jun 1 2011: IE: pages , requests iPhone: pages , requests
May 16 2011: IE: pages , requests iPhone: pages , requests
Apr 30 2011: IE: pages , requests
Apr 15 2011: IE: pages , requests
Mar 29 2011: IE: pages , requests
Mar 15 2011: IE: pages , requests
Feb 26 2011: IE: pages , requests
Feb 11 2011: IE: pages , requests
Jan 31 2011: IE: pages , requests
Jan 20 2011: IE: pages , requests
Dec 28 2010: IE: pages , requests
Dec 16 2010: IE: pages , requests
Nov 29 2010: IE: pages , requests
Nov 15 2010: IE: pages , requests
sponsors :
Google ,
Mozilla ,
New Relic ,
O’Reilly Media ,
Etsy ,
dynaTrace ,
Instart Logic ,
Catchpoint Systems ,
Fastly ,
SOASTA mPulse ,
Hosting Facts ,
TheBestVPN ,
and
Akamai
.
powered by WebPagetest