by 147 on 9/8/21, 5:30 PM with 130 comments
by jeremyepling on 9/8/21, 9:15 PM
2.1 We're starting work on this in the next couple of months. We plan to ship it in early 2022.
2.2 We want to speed up the pace of GitHub Enterprise Server releases, but I don't have more info to share.
2.3 We're looking at ways to not require a GHEC account or "unified" license.
2.4 The limits are much higher with the GitHub hosted runners, but this is a current limit of self-hosted runners.
3.1 It's on our backlog. No date to share.
3.2 I haven't heard this before and thanks for sharing the scenario. We'll think about it.
3.3 This will ship in October.
3.4 We're doing some performance optimizations for GHES 3.4 that should fix this.
3.5 This shipped recently - https://github.blog/changelog/2021-08-25-github-actions-redu...
3.6 We have a couple API improvements coming later this year.
3.7 We're looking into this, but no dates to share.
We're dedicated to making Actions a great experience. As you would assume, I'm very excited about the future of Actions and getting feedback like this helps us make it better.
by Octabrain on 9/8/21, 6:45 PM
I known is a fairly recent platform but I was expecting much more compared to what other services offer.
by chad_oliver on 9/8/21, 7:36 PM
In our case we've got some cypress tests that we want to run on a specific on-premise server every time we create a pull request. They take about 20 minutes to run, and we're creating a lot of pull requests, so you have to carefully check what github actions are executing before you create a pull request or push new changes to it. I'd love support for proper queues like what teamcity and other CI systems have.
by judge2020 on 9/8/21, 6:26 PM
- can't manually delete cache (although they're considering changing that[0])
- only saves cache at the end of a workflow, and only if the workflow succeeds. This could be solved with CircleCI's approach of having a save-cache step and a restore-cache step.
- Cache is super slow for self-hosted runners, so it makes more sense to have a local cache instead of using the action
- only 5gb of storage size. This was supposed to be increased via billable cache storage[1], but it's been on the backburner since July 2020
- In addition to the above, you can't use a different storage backend on the official action (which would allow storing over 5GB of cache via your own storage). The best workaround is to use a user-provided action which utilizes the s3 api[2].
0: https://github.com/actions/cache/issues/632
1: https://github.com/github/roadmap/issues/66
2: https://github.com/actions/cache/issues/354#issuecomment-854...
2>
by sofixa on 9/8/21, 6:55 PM
I really wonder why would anyone self-host GitHub. Gitlab has a much more feature rich, mature and cheap ( there's a perfectly usable free version) offering. Yeah, someone might prefer Github's UX, but is it really worth it to pay for a worse product?
by boleary-gl on 9/8/21, 9:32 PM
2.1 Caching isn’t available: GitLab has this everywhere.
2.2 GitHub Enterprise Server is behind GitHub Enterprise Cloud: GitLab ships the same code to GitLab.com as it does to our self-managed customers. This was a tough decision but has a lot of benefits...the central being feature parity and scalability for self-managed folks
2.3 Using Public GitHub.com Actions: This is a symptom more than the problem itself - relying on third-party plugins for build jobs is scary, and leads to many of the same issues we’ve seen in the Jenkins ecosystem - easy to get started, hard to maintain.
2.4 Dockerhub pull rate limiting: for self-hosted runners, you can use a registry mirror or Dependency Proxy to reduce your number of pulls from Docker Hub. The key is the entire platform has to be there to enable the right workflows.
3.1 No dropdowns for manually triggered jobs: GitLab also doesn’t have drop downs, but does have the ability to pre-fill these values.
3.2 Self-hosted runner default labels: I think this is also more of a symptom than a problem. 3.3 Being able to tag and use runners for specific tasks is key - so I understand the frustration and we’ve spent a lot of time on this.
3.4 You can’t restart a single job of a workflow: You can do this with GitLab.
3.5 Slow log output: I haven’t seen this be a problem, and is a benefit of our scalability features being built into the self-managed code.
3.6 You can’t have actions that call other actions: There are lots of ways to relate pipelines (parent/child, triggers. etc.) in GitLab.
3.7 Metrics and observability: The GitLab runner has Prometheus build in, and the dashboards we use to manage GitLab.com are partially public: https://dashboards.gitlab.com
3.8 Workflow YAML Syntax can confusing: This can be really hard to get right. I learned to stop worrying and love the YAML long ago, and I know we’ve got through a lot of iterations to try and get this right.
I'd love to know where folks think I got this assessment wrong. And is there value in writing more about it?
(edited for line spacing)
by oefrha on 9/8/21, 6:43 PM
by cebert on 9/8/21, 6:58 PM
by thinkafterbef on 9/8/21, 6:52 PM
Actually, most points in the article are the basis on why we created BuildJet.
We initially tried to solve these annoyances by creating a CI with speed and the YAML config as a USP. We got 4x speed and a much better YAML config structure, but despite these improvements we noticed that it people had a mental barrier to migrate to a new unknown CI.
But like OP we always enjoyed the experience of using GitHub Actions, so with this in mind. We decided to build BuildJet for GitHub Action[1] that uses the same infrastructure but plugs right into Github Action as a "self-hosted" runner, which is automatically set up for you with OAauth. This resulted on average a 2x speed improvement for half the cost(due to us being close to the metal). Easy to install and easy to revert.
by ChrisMarshallNY on 9/8/21, 8:24 PM
When I left my last job, and started working on my own, I set up things like CI/D, JIRA, Jenkins, etc. These were the bread and butter for development in my old shop.
But they are "Concrete Galoshes"[0], and work very well for teams, as opposed to ICs. As a single developer, working alone, the infrastructure overhead just slowed me down, and, ironically, interfered with Quality.
When GH Actions were first announced (I can't remember, but they may have been beta, then), I set up several of them, on my busier projects. They worked great, until I started to introduce some pivots, and I realized that there was actually no advantage to them. I ran the tests manually, anyway, and the Actions just gave me one more thing to tweak. It was annoying, getting the failure messages, when I knew damn well, the project was fine. I'd just forgotten to tweak the Action. I introduce frequent changes, in my work, and that is great.
[0] https://littlegreenviper.com/miscellany/concrete-galoshes/
by epage on 9/8/21, 8:42 PM
Among my small, open source work, probably my biggest complaint is actions running in forks. Wastes a lot of resources on their side and limits my concurrent runners for projects in my personal space. For companies, depending on the setup, this would eat their compute minutes.
Also annoying that PR actions can't post to the PR. I can understand there are security limitations but it makes it so a lot of nice features don't exist for most people.
by simonw on 9/8/21, 6:52 PM
by staticassertion on 9/8/21, 7:53 PM
Once you get into more complex things - like building docker images, storing into an artifact repository, baking amis, running integration or end to end tests, etc, it can be a pain.
It was a great place for us to start but we've since moved to BuildKite.
by bob1029 on 9/8/21, 6:26 PM
Everything else is managed via a custom tool we use for packaging & deploying our product.
Even our simple "run this 1 build command and ensure exit code == 0" action seems to have a semi-weekly issue like stuck "waiting for status" and other unexplained failures throughout. We don't want to put any more eggs into that particular basket right now.
by lmeyerov on 9/8/21, 6:27 PM
Our main gotchas are roughly:
- GH-hosted runners have too little RAM/HD for big docker software. They push you to self-hosted runners for that, which is fine in theory, but GHA/Azure doesn't actually support serverless runners, so that falls flat in practice. We don't want to be turning machines on/off, that's GHA's job. We experimented with GHA -> Packer -> Azure for serverless, but it was slow and Packer frequently leaves zombie machines, so we went back to tweaking the low-RAM runners provided by our enterprise plan.
- Security: We want contactors etc. to be able to run limited GHA CI jobs and use that quota, but not higher-trust GHA CD ones. This is tricky at a configuration level. Ex: It seems like we'd need to funny things like main repo for CI w/ CI secrets, and a separate repo for CD w/ CD secrets, and only give untrusted folks access to the CD-cred repo. We've thought of other possibilities as well, but in general, it's frightening.
- Big Docker images: We do spend more time than I'd like messing with optimizing Docker caching as GPU containers are embarrassingly big (we use dockerhub vs github's due to sizes/pricing/etc), think both multi-stage containers + multi-step jobs (monorepo/microservices). I think they're in a good position to speed that up!
I'm optimistic about these, but tricky to align with MS/GH PM personal team priorities :)
by smcleod on 9/9/21, 2:06 AM
The biggest issue I have is around self-hosted runners.
1. There's no official auto-scaling runner option, so even if you're paying Github (aka Microsoft) for Enterprise - they're not going to support your auto-scaling EKS/GKE/EC2/whatever runners.
2. You can't register self-hosted runners without a Personal Access Token - the key word being _Personal_. Your automation code for provisioning runners should not rely on an individuals Github access token just to register, they need to have a system like GitLab has where you can generate a registration token per-organisation/team/repo that allows you to programmatically register runners.
by thedougd on 9/8/21, 10:54 PM
My wishlist item would be more variants of Windows server versions so that we could build Windows containers for more versions of Windows. I realize the fault lies with Windows containers pinning the container base version to the host version, but I'm still stuck with the burden.
I think GitHub Actions got the model correct, using everything as events to trigger any number of workflows. This is far simpler to maintain than a single workflow with conditionals and wait states that you see with other systems.
by elpakal on 9/8/21, 8:11 PM
by gitgud on 9/8/21, 11:13 PM
The verbosity of accessing output has the added benefit of making it much clearer that the 2 workflow steps are tightly interdependent on each other.
by chad_oliver on 9/8/21, 7:29 PM
As of August 25, you can! https://github.blog/changelog/2021-08-25-github-actions-redu...
by Waterluvian on 9/9/21, 2:17 AM
The problem is that I cannot make the image build action happen IFF both testing actions pass. I had to combine all three actions into one.
It works. But now there’s a “skipped” step that’s skipped 99% of the time and makes no sense for a lot of PRs. It also means I have a Frankenstein monster action that does three long lists of very different things. All just so I can make 3 depend on 1 and 2.
The other problem is that to develop and Test an action, I have to just push to origin a thousand times. My kingdom for a CI system that _trivially_ enables me to install a single one-liner program that lets me locally test my actions at near 1:1 compatibility.
by Aeolun on 9/8/21, 11:04 PM
by m_a_g on 9/8/21, 6:43 PM
With the default GITHUB_TOKEN, you can't push to protected branches. If you decide to use personal access tokens, you can push to protected branches, BUT that will trigger other workflows. That can cause an infinite loop of workflows.
We still couldn't figure out how to push code to a protected branch without triggering the same/other workflows.
by BugsJustFindMe on 9/8/21, 9:35 PM
by Dowwie on 9/9/21, 3:38 PM
by AJRF on 9/8/21, 8:40 PM
* Full clean build including dependencies (support Carthage, Cocoapods, SPM)
* Running multiple test suites that takes maybe 5+ hours for full suite?
* Running simulators for screenshot testing
by shadycuz on 9/8/21, 8:40 PM
It let's you run actions on top of Jenkins.
by arminiusreturns on 9/8/21, 8:20 PM
by throwaway20371 on 9/8/21, 7:57 PM
Oh jesus christ. I feel for you dude.
We evaluated GHA, and we still are trying to use it, but there is a barrage of problems and limitations, including cost, lack of functionality, and technical issues. It's really only suitable (at scale) for linting, or generating Changelogs, or something else trivial. I use it in my OSS projects to run tests, and it's okay for that (though impossible to just tail a build log when it's large)
Drone.io is still an amazingly effective system that matches GHA (and has _crazy_ features like build parameters) but is more flexible. Of course you'll have to pay for commercial licenses, but if it's between paying for GHA or Drone, I highly recommend Drone instead. Drone is stupidly easy to maintain (infrastructure-wise).