from Hacker News

Kaggle Datasets – Discover and analyze open data

by benhamner on 3/26/18, 5:53 AM with 36 comments

  • by benhamner on 3/26/18, 7:35 AM

    Our goal with Kaggle Datasets is to provide the best place to publish, collaborate on, and consume public data.

    As a data publisher, you have an easy way to publish data online, see how it's used, and interact with the users of the data. You can create the dataset via a simple web interface, and update it through the interface or an API. We automatically version these updates under the hood.

    As a data consumer, you can browse the data online and download it (through the web or an API). You can see the code and insights others have generated on the data through Kaggle Kernels (hosted, versioned IPython notebooks that run in Docker containers). You can fork their code to get started on the data, or start coding from scratch on your own analysis. If you find improvements that could be made to the metadata (dataset/file/column-level descriptions), you can make those directly.

    We're rapidly iterating on this product and expanding it's functionality, and would love any feedback and suggestions.

  • by QasimK on 3/26/18, 6:44 AM

    How about you let me download them without creating an account before calling them “public”?

  • by antirez on 3/26/18, 9:22 AM

    This is gold. When I wrote the NeuralRedis module I had so much fun downloading a few random datasets from Kaggle and wrap it in a few lines of Ruby script to check what the results were in terms of predictions. Normally the data is very high quality, the format well documented, and so forth. However make sure to check the license for the details depending on what use you plan to do.

  • by Radim on 3/26/18, 5:06 PM

    What happens when the company changes direction? If there's a shift of priorities, an internal restructuring, a "strategic startup pivot", an acquisition?

    Not to assume bad faith on Kaggle's part, but we got burned one too many times with private companies pushing their proprietary ("open") platforms for gobbling up data. The "it's free! just create an account — data lock-in — gap after project death/monetization" pattern leaves me a little cynical.

    It's awesome that resources like these exist, but I'd be more comfortable paying attention if this was hosted as raw data somewhere (Github?), with a clear licensing and access model.

  • by neuromantik8086 on 3/27/18, 1:52 AM

    The Awesome Public Datasets Github repo [1] also constitutes a good effort at organizing all of the open data out there that people can play around with.

    [1] https://github.com/awesomedata/awesome-public-datasets

  • by metakermit on 3/26/18, 2:00 PM

    Wonderful, thanks for sharing this! It's useful that the kernels people have submitted are there as well and that there is a HN-style upvoting mechanism.

    As an aside – I'm really curious to explore the datasets with "fake" in the title :)

    https://www.kaggle.com/datasets?sortBy=relevance&group=publi...

  • by cosmic_ape on 3/26/18, 7:59 AM

    It would help if the datasets were categorized by data type. Timeseries, multilabel, etc...

  • by socksy on 3/26/18, 5:01 PM

    Is there an announcement of some kind of change? Are they still owned by Google? Or is this the thing where sometimes existing solutions will hit the front page of HN? :)

  • by naushit on 3/26/18, 1:15 PM

    Any plan to share same data/files using IPFS?