from Hacker News

OpenFlights – airport and airline data

by cyberlab on 4/27/21, 2:25 PM with 60 comments

  • by amatecha on 4/27/21, 8:02 PM

    You can get free enterprise access to a lot of the major flight tracker services by setting up an ADSB receiver and feeding the data to them. Basically they give you full access to everything as if you were paying for the top tier of their services, because you're helping increase the coverage of their data. A few such services:

    https://www.flightradar24.com

    https://flightaware.com/

    https://www.radarbox.com/

    https://skyscanworld.com/

    There's also https://www.adsbexchange.com/ which doesn't filter their data (probably much to the chagrin of various businesses and governments). If you see/hear a weird plane above and you can't find it on the commercial services above, check ADSB Exchange.

  • by Fomite on 4/27/21, 3:43 PM

    Briefly got interested, then hit "Warning: The third-party that OpenFlights uses for route data ceased providing updates in June 2014. The current data is of historical value only."

  • by phsource on 4/27/21, 3:46 PM

    It's ridiculous that the only original source of this data, the IATA [0], charges $700+ for this list, so kudos to OpenFlights.

    I can't stress just how important (and how hard) it is to get a great source of data for airports -- I've now built 3 travel-related projects (the latest, Wanderlog [https://wanderlog.com], keeps people's flight reservations, so uses it for an autocomplete), and it's been a key building block for all of them.

    The main datasets we use are:

    - OpenFlights [1]: mentioned in this post, but this dataset was great since it had timezone too.

    - OurAirports [2]: no timezone here, but the "type" and "scheduled_service" columns in this dataset are essential. "Type" lets you distinguish between small/medium/large airports, and "scheduled_service" lets you easily filter out airports without real flights (which you often might not care about).

    - Random other GitHub Gist [3]: I have no idea where this data comes from, but it was surprisingly complete and has a few golden nuggets like "num_flights" and "runway_length" in addition to "timezone". The presence of a "woeid" suggests Yahoo-related origins, but it's hard to be sure.

    - We now supplement this with airports from autocomplete APIs like Skyscanner's, because they're still the most up-to-date.

    Long story short, it'd be AWESOME to have one complete, updated database with all this data in one place. This kind of data really should be public and a public service, but until then it's unfortunately up to the community.

    [0] https://www.iata.org/en/publications/store/airline-coding-di...

    [1] https://github.com/jpatokal/openflights/

    [2] http://ourairports.com/data/

    [3] https://gist.github.com/tdreyno/4278655

  • by wyozi on 4/27/21, 3:10 PM

    Downloadable airport data on OpenFlights tends to be quite dated, missing notably e.g. the Berlin Brandenburg Airport. OurAirports (https://ourairports.com/) has a slightly different format but the data there is significantly more recent.

    source: started with OpenFlights but had to switch to OurAirports for my project https://flightnotebook.com

  • by einpoklum on 4/27/21, 3:02 PM

    The US Bureau of Transport statistics provides historic flight schedule and actual flight performance data in CSV tables:

    http://www.transtats.bts.gov/DL_SelectFields.asp?Table_ID=23...

    But it's cumbersome to work with.

    I am working (on and off) on a DBMS benchmark based on this data. As part of that endeavor, I have a script which:

    * Automates downloading the CSVs.

    * Creates an appropriate SQL database schema.

    * Performs a bit of rudimentary cleaning (e.g. invalid character codes; optional)

    * Loads the CSV files into the database.

    So that, from the command-line, you could get the flight on-time performance data by merely typing in something like:

      /path/to/usdt-ontime-tools/scripts/setup-usdt-ontime-db -r  -db-name ontime --first-year 2019 --last-year 2020
    it's available within this repository:

    https://github.com/eyalroz/usdt-ontime-tools

    the caveat is that, for now, the only DBMS supported directly is MonetDB: https://www.monetdb.org/ , a FOSS analytics-oriented columnar DBMS.

    An adaptation of the script for other systems (MySQL/Maria, PostgreSQL) should be straightforward, since the commands are SQL'ish after all. If you're interested in that, open an issue or write me.

  • by kingsloi on 4/27/21, 5:48 PM

    For anyone interested, the nice guys at https://aviation-edge.com supplied me access to their flight API so I can track how many flights fly directly over my little community in Gary Indiana: https://millerbeach.community

    I wish I was able to track more frequently than every 15 minutes (free version api max, etc), because some aircraft pass overhead before they're picked up, so it's not the most accurate, but a rough figure to/from O'Hare, Midway, and Gary

  • by JamilD on 4/27/21, 6:10 PM

    A few years ago I tried writing a Python wrapper around SABRE's API to get pricing, route, and schedule data, which seemed to work reasonably well. It likely doesn't work anymore, but it was a fun exercise. https://github.com/Jamil/sabre_dev_studio

    I wish I had access to the GDS data to get realtime seat/award availability, but I couldn't find any pricing information to get that information through Sabre's API.

    Does anyone know how much that costs, or if there are any services which provide it as an API? I use ExpertFlyer for personal use, but ideally I'd want to get that information at the source…

  • by asix66 on 4/27/21, 6:40 PM

    Try the "crowdsourced" ADS-B Exchange site, which shows unfiltered flight data. [0] For more info, check their FAQ.

    Live data: https://globe.adsbexchange.com

    [0] https://www.adsbexchange.com

  • by leugim on 4/27/21, 2:51 PM

  • by radelaine on 4/27/21, 6:59 PM

    For those who want to jump in and query this dataset, I uploaded it here: https://bit.io/boyd/airports

    I'm still working on bit.io and would love feedback so hit me.

  • by atbpaca on 4/28/21, 1:14 AM

    Interesting! In addition to the links already shared, I use OpenTravelData, available at https://github.com/opentraveldata/opentraveldata, which consolidates airport information from different sources, but also data on aircrafts, airlines, etc.

  • by jjwiseman on 4/27/21, 7:06 PM

    If you need airport or airspace data, openaip seems to be decent: http://www.openaip.net/

    For some reason they make you register in order to download the data, and the site is a bit confusing, but the data seems good.

  • by rkwasny on 4/28/21, 1:02 PM

    In the past I wanted to build a system that checks flight paths and will tell you what kind of plane noise you can expect in the area where you want to buy a house.

    Anyone build it yet/ needs something like this?

  • by ALittleLight on 4/27/21, 10:44 PM

    Why don't airlines provide a good free API for flights and reservations? I would think they would want developers to help make accessing their offerings and buying them easier.

  • by gbrindisi on 4/27/21, 4:07 PM

    does anyone know any current dataset I could query to check historical routes status?

    Since the pandemic I've found plenty of airlines selling tickets and systematically cancel the flight a few days before. I was looking to scrape some data to avoid this kind of unreliable flights.

  • by Jemm on 4/28/21, 12:16 PM

    typing in Toronto does not return anything, neither do many of the smaller airport ICAO I trired.

  • by rumblestrut on 4/27/21, 7:58 PM

    I wish there was open train data.