from Hacker News

Ask HN: Is GitHub down?

by mikebonnell on 6/29/23, 5:42 PM with 184 comments

Not loading for me at all, but status page shows green across the board.

  • by edgyquant on 6/29/23, 5:53 PM

    Everytime github goes down, and my push/pull is rejected, I immediately assume they’ve discovered I’m incompetent and fired me. And I’m the head of engineering at my company.

  • by klysm on 6/29/23, 5:46 PM

    Seems to be a fairly catastrophic failure. https://github.com/ fails to load. https://www.githubstatus.com/ shows all green as of this writing. Nothing on the twitter yet https://twitter.com/githubstatus

    edit: The outage is now acknowledged on the status page https://www.githubstatus.com/

    edit: EU folks appear to have things working so it looks like a regional network fault

  • by hn8305823 on 6/29/23, 5:49 PM

    140.82.113.0/24 is visible in the global routing table:

      route-views>sh bgp 140.82.113.0
  BGP routing table entry for 140.82.113.0/24, version 62582026
  Paths: (19 available, best #4, table default)
    The route is verified by RPKI so it's not a route hijack.

    Edit: deleted traceroute

  • by mbreese on 6/29/23, 5:53 PM

    https://www.githubstatus.com/

    Just flipped to red.

    See here: https://www.githubstatus.com/incidents/gqx5l06jjxhp

    >Investigating - We are currently experiencing an outage of GitHub products and are investigating.

    >Jun 29, 2023 - 17:52 UTC

  • by lucb1e on 6/29/23, 5:59 PM

    https://codeberg.org open source GitHub without Microsoft (it's a German non-profit). You can also host your own lightweight https://forgejo.org instance

    In case anyone questions whether centralizing literally everything onto GitHub is a good idea, at least as a mirror for things you depend on

  • by ChrisArchitect on 6/29/23, 5:49 PM

  • by stefanos82 on 6/29/23, 5:46 PM

    So down right now...I wonder why they still use https://www.githubstatus.com/ that reports everything is alright when it's not!

  • by lucb1e on 6/29/23, 6:03 PM

    To all the "same" and "not for me" posters: the very least you could add is a location

  • by hejcloud on 6/29/23, 6:04 PM

    I feel sympathy for all the engs at companies I've implemented CI/CD based on Gh Actions in recent years. It's not like I didn't tell them that Github showed to be somewhat unreliable in the recent years and in contrast to their claim "it's just the build pipeline, not the product" I think it is a horrible incident if you're not able to deploy to production and have barely any ad-hoc backup.

    I'm always evangelizing Argo or Flux and some self-hosted Gitlab or gitea, but seems like they all prefer to throw their money at Github as of now.

  • by ActualHacker on 6/29/23, 6:02 PM

    Since nobody can work, I'll just leave this here: "I must have put a decimal point in the wrong place or something. Shit! I always do that. I always mess up some mundane detail."

  • by charlieyu1 on 6/29/23, 5:47 PM

    Loads for me

    Unless it is cached

    Edit: I could even login

  • by whitewingjek on 6/29/23, 5:47 PM

    Yes, github status says all is operational but hacker news is faster. Go figure.

  • by gaoshan on 6/29/23, 6:03 PM

    My team ran some code that crushed our Github actions very shortly before this outage. Nervous laughter around the office that it was us.

  • by sdsd on 6/29/23, 6:07 PM

    I am laughing so hard right now. My best friend mocked me for using git.lain.faith to host my code, saying it wasn't reliable. Well, well, well. In the last year GitLain hasn't gone down once.

    I know he was still right in a way, who knows when git.lain.faith will just disappear. But still. I'm going to send some texts to bother him right now, hahaha.

  • by onionisafruit on 6/29/23, 6:00 PM

    I heard from somebody at GitHub that this one will make a good incident report. No other details or estimates for recovery time.

  • by Macuyiko on 6/29/23, 6:07 PM

    Update

    We have identified the root cause of the outage and are working toward mitigation Posted 4 minutes ago. Jun 29, 2023 - 18:02 UTC

  • by oblongatacur on 6/29/23, 6:03 PM

    I noticed Github's OIDC token changed about a half an hour ago. Security incident?

  • by billy1kaplan on 6/29/23, 5:54 PM

  • by jesterman on 6/29/23, 5:52 PM

  • by renonce on 6/29/23, 6:02 PM

    Not down for me accessing from Hong Kong. I suspect this is a regional outage.

  • by rvz on 6/29/23, 6:26 PM

    Unsurprising. There is at least one outage with GitHub every single month. [0]

    [0] https://news.ycombinator.com/item?id=35967921

  • by turtleyacht on 6/29/23, 5:49 PM

    Won't load for me.

    https://githubstatus.com shows all green, but it's not the case...

  • by kashnote on 6/29/23, 5:49 PM

    Are these status pages updated manually? At the very least it should be able to detect that the home page doesn't even load and turn itself red.

  • by gwbas1c on 6/29/23, 6:17 PM

    Yaay! I just pushed and my new commit showed up in CI!

  • by gwbas1c on 6/29/23, 5:45 PM

    I can't push from my desktop, and https://github.com/ spins in the browser

  • by computershit on 6/29/23, 5:43 PM

    Yup. Totally down, happened right as I opened a PR.

  • by acim on 6/29/23, 6:28 PM

    I just noticed that artifacts download didn't work although the web site was up. There was some varnish proxy error.

  • by numbsafari on 6/29/23, 5:52 PM

    If they used AI to rebuild their system and migrated it Azure, I bet they would stop having all the problems.

  • by joshstrange on 6/29/23, 5:45 PM

    Yep, looks like it's down. Can't pull/push and can't even get the web to load at all.

  • by aftaylor2 on 6/29/23, 5:45 PM

    Unable to load Github website or push using git-remote-https as of the last several minutes.

  • by MorningInfidel on 6/29/23, 5:45 PM

    Same. Spins for a while then dreaded `ERR_CONNECTION_TIMED_OUT` chrome error page.

  • by saintblasphemer on 6/29/23, 6:11 PM

    Looks like we are back online.

  • by distortionfield on 6/29/23, 5:52 PM

    Down for me as well, Operation Timed Out errors on all attempted SSH connections

  • by Gordonjcp on 6/29/23, 6:00 PM

    No it isn't, it's working absolutely fine and has been all afternoon.

  • by ny711 on 6/29/23, 6:26 PM

    This should be a weekly ASK HN; seems to happy pretty frequently at this point

  • by circuit10 on 6/29/23, 6:01 PM

    It works for me at the moment

  • by JohnMakin on 6/29/23, 5:49 PM

    First noticed when trying to pull a helm chart - get a 503 backend error page.

  • by squalo on 6/29/23, 6:51 PM

    Not a single day passes without a MAJOR outage in a Microsoft owned service.

  • by escape-big-tech on 6/29/23, 6:11 PM

    often people point out how unreliable self-hosted services are, well, hosted services are just as unreliable if not more.

    this, ladies and gentlemen, is why you should always self host critical infrastructure

  • by coder9874 on 6/29/23, 5:47 PM

    Yep it's down. Why do they even bother with the status page

  • by connorgutman on 6/29/23, 6:06 PM

    Up in Africa (Morocco).

  • by datalus on 6/29/23, 6:02 PM

    And just when I was about to get into flow state...

  • by duckkg5 on 6/29/23, 5:53 PM

    Status page updated showing all red https://www.githubstatus.com/

  • by GrumpyNl on 6/29/23, 5:45 PM

    Same here, it cant send a verification sms.

  • by jetpackjoe on 6/29/23, 5:47 PM

    Its down for me

  • by ukrainiancrab on 6/29/23, 6:04 PM

    Cannot use oauth2 in algoexpert :/

  • by wcarss on 6/29/23, 5:45 PM

    I can't even load the status page.

  • by Xeamek on 6/29/23, 6:03 PM

    looking on the bright side, at least we'll get an interesting post-mortem to read in a day or two.

  • by ushakov on 6/29/23, 5:47 PM

    Actions won't start for me

  • by mydriasis on 6/29/23, 6:03 PM

    Yes. Can't even pull ;(

  • by ughitsaaron on 6/29/23, 6:28 PM

    Good time to grab a beer!

  • by tednotfred on 6/29/23, 5:48 PM

    Down for me right now.

  • by sparc24 on 6/29/23, 6:05 PM

    Azure Strikes Again!

  • by themusicgod1 on 6/29/23, 6:31 PM

    It shouldn't matter. Nobody should be using github post-2018.

  • by facu17y on 6/29/23, 5:59 PM

    seems to be a network issue, not a service issue

  • by almost_usual on 6/29/23, 5:50 PM

    Works for me?

  • by etimberg on 6/29/23, 5:43 PM

    Same for me

  • by abe-101 on 6/29/23, 5:49 PM

    Same here

  • by ActualHacker on 6/29/23, 5:46 PM

    Yep

  • by saintblasphemer on 6/29/23, 5:44 PM

    Same here.

  • by rk32 on 6/29/23, 6:01 PM

    its all down currently

  • by adpirz on 6/29/23, 5:43 PM

    Same