from Hacker News

Deep learning job postings have collapsed in the past six months

by bpesquet on 8/31/20, 11:27 AM with 264 comments

by eric_b on 8/31/20, 1:35 PM
I've worked in lots of big corps as a consultant. Every one raced to harness the power of "big data" ~7 years ago. They couldn't hire or spend money fast enough. And for their investment they (mostly) got nothing. The few that managed to bludgeon their map/reduce clusters in to submission and get actionable insights discovered... they paid more to get those insights than they were worth!
I think this same thing is happening with ML. It was a hiring bonanza. Every big corp wanted to get an ML/AI strategy in place. They were forcing ML in to places it didn't (and may never) belong. This "recession" is mostly COVID related I think - but companies will discover that ML is (for the vast majority) a shiny object with no discernible ROI. Like Big Data, I think we'll see a few companies execute well and actually get some value, while most will just jump to the next shiny thing in a year or two.
by AznHisoka on 8/31/20, 7:10 PM
According to data from Revealera.com, if you normalize the data, the % of job openings that mention 'deep learning' has actually remained stable YoY: https://i.imgur.com/sDoKwD0.png
* Revealera.com crawls job openings from over 10,000 company websites and analyzes them for technology trends for hedge funds.
by simonw on 8/31/20, 5:25 PM
Something I've learned: when non-engineers ask for an AI or ML implementation, they almost certainly don't understand the difference between that and an "algorithmic" solution.
If you solve "trending products" by building a SQL statement that e.g. selects items with the largest increase of purchases this month in comparison to the same month a year ago, that's still "AI" to them.
Knowing this can save you a lot of wasted time.
by ineedasername on 9/1/20, 12:49 AM
Most data-related problems, or extraction of knowledge from data, simply doesn't benefit from Deep Learning.
In my experience, what many organizations lack is simple but high-quality "Business Analytics": Reporting & dashboards are developed that look good but jam too much information together. It is often the wrong information:
Something is requested, and the developer develops exactly what was asked. The problem is that it wasn't what was needed because the person making the request couldn't articulate the question in the same terms the developer would understand. The request will say "Give me X & Y" when the real question is "I want to understand the impact of Y on X". The person gets X & Y, looks at it every day in their dashboard, and never sees much that is useful. The initial request should always be the start of a conversation, but that often doesn't happen. A common result are people in departments spending tons of time in Excel sorting, counting, making pivot tables, etc., when all of that could be automated.
This is part of the reason why companies often go looking for some new "silver bullet" to solve their data problems. They don't have the basics down, and don't understand the data problems well enough to seek out a solution.
by The_rationalist on 8/31/20, 1:11 PM
I observe the state of the art on most Nlp tasks since many years: In 2018,2019 there was huge progress made each year on most tasks. 2020,except for a few tasks have mostly stagnated... NLP accuracy is generally not production ready but the pace of progress was quick enough to have huge hopes. The root cause of the evil is: Nobody has build upon the state of the art pre trained language: XLnet while there are hundreds of declinaisons of BERTs. Just because of Google being behind it, if XLnet was owned by Google 2020 would have been different. I also believe that pre trained language have reached a plateau and we need new original ideas such as bringing variational autoencoder to Nlp and using metaoptimizers such as Ranger.
The most pathetic one is that: Many major Nlp tasks have old SOTA in BERT just because nobody cared of using (not improving) XLnet on them which is absolute shame, I mean on many major tasks we could trivially win many percents of accuracy but nobody qualified bothered to do it,where goes the money then? To many NIH papers I guess.
There's also not enough synergies, there are many interesting ideas that just needs to be combined and I think there's not enough funding for that, it's not exciting enough...
I pray for 2021 to be a better year for AI, otherwise it will show evidence for a new AI progress winter
by EForEndeavour on 8/31/20, 12:38 PM
While this sounds plausible and has a lot of "prior" credibility coming from someone as central to deep learning as François Chollet, I'd love to see corroborating signal in actual job-posting data, from LinkedIn, Indeed, GlassDoor, etc. Backing up this kind of claim with data is especially important given the fact that the pandemic is disrupting all job sectors to varying degrees.
As you can imagine, searching Google for "linkedin job posting data" doesn't work so great. The closest supporting data I could find is this July report on the blog of a recruiting firm named Burtch Works [1]. They searched LinkedIn daily for data scientist job postings (so not specifically deep learning) and observed that the number of postings crashed between late March and early May to 40% of their March value, and have held steady up to mid-June, where the report data period ends.
There's also this Glassdoor Economic Research report [2], which seems to draw heavily from US Bureau of Labor Statistics data available in interactive charts [3]. The most relevant bit in there is that the "information" sector (which includes their definitions of "tech" and "media") has not yet started an upward recovery in job postings, as of July.
[1] https://www.burtchworks.com/2020/06/16/linkedin-data-scienti...
[2] https://www.glassdoor.com/research/july-2020-bls-jobs-report...
[3] https://www.bls.gov/charts/employment-situation/employment-l...
by supergeek133 on 8/31/20, 4:41 PM
I feel like it was also a classic case of running before we could crawl. Jumping from A to Z before we could go from 0 to 1.
I work at an Residential IoT company, there are quite a few really valid use cases for Big Data and even ML. (Think about predictive failure).
We hired more than one expensive data scientist in the past few years, and had big strategies more than once. But at the end of the day it's still "hard" to ask a question such as "if I give you a MAC Address give me the runtime for the last 6 months".
We're trying to shoot for the moon, when all I've ever asked is I want an API to show me indoor temp for particular device over a long period.
by joelthelion on 8/31/20, 1:01 PM
Meh, only for people who bought into the hype without real use cases. Which I agree may be numerous.
In my company though, we've been applying DL with great success for a few years now, and there are at least five years of work remaining. And that's not spending any time doing research or anything fancy: just picking the low-hanging fruit.
by bane on 8/31/20, 7:47 PM
I managing some teams right now that do a mix of high-end ML stuff with more prosaic solutions. The ML team is smart, and pretty fast with what they do, but they tend to (as many comments here have mentioned) focus on delivering only PhD level work. This translates into taking simple problems and trying to deorbit the ISS through a wormhole on it rather than just getting something in place that answers the problem.
In conjunction with this, it turns out 99% of the problems the customer is facing, despite their belief to the contrary, aren't solved best with ML, but with good old fashioned engineering.
In cases where the problem can be approached either way, the ML approach typically takes much longer, is much harder to accomplish, has more engineering challenges to get it into production, and the early ramp-up stages around data collecting, cleaning and labeling are often almost impossible to surmount.
All that being said, there are some things that are only really solvable with some ML techniques, and that's where the discipline shines.
One final challenge is that a lot of data scientists and ML people seem to think that if it's not being solved using a standard ML or DL algorithm then it isn't ML, even if it has all of the characteristics of being one. The gatekeeping in the field is horrendous and I suspect it comes from people who don't have strong CS backgrounds wrapping themselves too tightly against their hard-earned knowledge rather than having an expansive view of what can solve these problems.
by softwaredoug on 8/31/20, 10:09 PM
There's a lot of what I call "model fetishism" in machine learning.
Instead of focusing our energies on the infrastructure and quality of data around machine learning, there's eagerness to take bad data to very high-end models. I've seen it again and again at different companies, usually always with disastrous consequences.
A lot of these companies would do better to invest in engineering and domain expertise around the problem than worry about the type of model they're using to solve the problem (which usually comes later, once the other supporting maturity pieces are in place)
by arcanus on 8/31/20, 12:25 PM
This is an anecdote with no data. And the entire global economy is in a recession, so the fact deep learning might have fewer job postings isn't particular notable.
I'll note that in my personal anecdote, the megacorps remain interested in and hiring in ML as much as ever.
by occamrazor on 8/31/20, 4:31 PM
Missing in the original chart/data: have ML/DL job postings decrease more or less than other comparable job categories (programming, business analyst, etc.)
by fnbr on 9/1/20, 3:35 PM
I am a DL researcher at a top industry lab.
I'm completely unsurprised by this. Regularly, at lunch, I'll ask my coworkers if they know of any DL applications that are making O($billions), and no one knows any outside of FAANG.
FAANG is making an insane amount of money due to DL. Outside of them though, I don't know who's making money here. When I was interviewing for jobs, there were a ton of startups that were trying to do things with DL that would have been better done with a few if statements and a random forest, and that had a total market size in the millions.
I think that, eventually, there'll be a market for this stuff, but I'm not convinced that it's anywhere near being widespread.
I was also a consultant before my current role. The vast majority of non-tech firms don't have their data in well organized + cleaned databases. Just moving from a mess of Excel sheets to Python scripts + SQL databases would have made a HUGE difference to the vast majority of clients I worked with, but even that was too big of a transformation.
Basically, everyone with the sophistication to take advantage of DL/ML already has the in-house expertise to do it. There's almost no one in the intersection of "Could make $$$ doing DL" && "Has the technical infrastructure to integrate DL".
by dcolkitt on 8/31/20, 6:14 PM
99% of the time you don't need a deep recurrent neural network with an attention based transformer. Most times, you just need a bare-bones logistic regression with some carefully cleansed data and thoughtful, domain-aware feature engineering.
Yes, you're not going to achieve state-of-the-art performance with logistic regression. But for most problems the difference between SOTA and even simple models is not nearly as large as you might think. And two, even if you're cargo-culting SOTA techniques, it's probably not going to work unless you're at an org with an 8-digit R&D budget.
by tomhallett on 8/31/20, 6:29 PM
I know very little about the DL/ML space, but as a full-stack engineer it feels like most companies have tried to replicate what FAANG companies do (heavy investment in data/ml) when the cost/benefit simply isn't there.
Small companies need to frame the problem as:
1) Do we have a problem where the solution is discrete and already solved by an existing ML/DL model/architecture?
2) Can we have one of our existing engineers (or a short-term contractor) do transfer learning to slightly tweak that model to our specific problem/data?
Once that "problem" actually turns into multiple "machine learning problems" or "oh, we just need todo this one novel thing", they will probably need to bail because it'll be too hard/expensive and the most likely outcome will be no meaningful progress.
Said in another way: can we expect an engineer to get a fastai model up and running very quickly for our problem? If so, great - if not, then bail.
ie: the solution for most companies will be having 1 part-time "citizen data scientist" [1] on your engineering team.
[1]: https://www.datarobot.com/wiki/citizen-data-scientist/
by kovac on 9/1/20, 1:19 AM
The way I see it, only those companies that had already been using a data oriented approach to business can really reap the benefits of ML. From a company's point of view, ML/AI should be a natural evolution of an existing tool set to better solve problems they have been trying to solve in the past using deterministic methods and then statistical methods, etc. Any other project that is diving right into ML is likely to fail because
1. There's no clear problem statement. They have never formulated one and now trying to bolt ML on to their decision making.
2. They don't have well catalogued data for engineers/scientists to work with because they never tried to do rigorous analysis of data before ML became a thing.
3. Managers have no idea how to deal with data driven insights. What if the results are completely unintuitive to them? Are they going to change their processes abruptly? What if the results are aligned with what they have been always doing? Is it worth paying for something that they have been doing intuitively for decades?
I'm not a data scientist. But the biggest complaint I hear from my colleagues is that they lack data to train models.
by lm28469 on 8/31/20, 1:10 PM
Isn't it the same pattern every 10 years or so for "AI" related tech ? Some people hype tech X as being a game changer - tech X is way less amazing than advertised - investors bail out - tech X dies - rinse and repeat.
https://en.wikipedia.org/wiki/AI_winter
by nutanc on 8/31/20, 2:05 PM
AI has a business problem.
Very few businesses I know actually have a deep learning problem. But they want a deep learning solution. Lest they get left out of the hype train.
by calebkaiser on 8/31/20, 12:57 PM
"This is evident in particular in deep learning job postings, which collapsed in the past 6 months."
Have they? Specifically, have they "collapsed" relative to the average decline in job listings mid-pandemic?
by whoisjuan on 8/31/20, 6:55 PM
Companies trying to add machine learning to everything they do like if that's going to solve all their problems or unlock new revenue streams.
80 or 90% of what companies are doing with machine learning results in systems with a high computing cost that are clearly unprofitable if seen as revenue impacting units. Many similar things can be achieved with low-level heuristics that result in way smaller computing costs.
But nobody wants to do that anymore. There's nothing "sexy" or "cool" about breaking down your problems and trying to create rule-based systems that addresses the problem. Semantic software is not cool anymore, and what became cool is this super expensive blackbox that requires more computer power than regular software. Companies have developed this bias for ML solutions because they seem to have this unlimited potential for solving problems, so it seems like a good long term investment. Everyone wants to take that bus.
Don't get me wrong. I love ML, but people use it for the stupidest things.
by ur-whale on 8/31/20, 12:42 PM
That may be true in the research arena (where Mr Chollet works), but I don't think that's the case in terms of where deep learning is actually applied in industry, nor will it be the case for years to come IMO.
It's just that much that needed to be invented has been invented and now it's time to apply it everywhere it can be applied, which is a great many place.
by insomniacity on 8/31/20, 12:28 PM
Some context, for those unfamiliar: https://en.wikipedia.org/wiki/AI_winter
by jungletime on 9/1/20, 1:07 AM
I've been using voice commands on my android phone, in situations where I can't use my hands. Most often all I want to do is.
1. Start and stop a podcast.
2. Play music
3. Ask for the time
The phone understands me, but then android breaks the flow, so I have to use my hands.
1. It will ask me to unlock the phone first? I have gloves and a mask on. It won't recognize my face, and my gloves don't register touches. Why do I have to unclock the phone to play music in the first place.
2. It gets confused on which app to play the music/podcast on. Wants to open youtube app, or spotify, and so on ...
3. Not consistent. I can say the same thing, and sometimes it will do one things, and another next time.
4. If I'm playing a video, and I want to show it full screen. I have to maximize and touch the screen. Why can't it play full screen be default.
by mijail on 8/31/20, 4:10 PM
My favorite joke on this is "The answer is deep learning, now whats the problem?"
by atsushin on 9/1/20, 1:31 PM
I'm currently a masters student and I'm rather glad I opted not to take a specialized degree such as Machine Learning, taking on computer science instead. All this discussion about DS, ML, AI (and even CS) becoming over-saturated has made me rather wary and I worry that I'm choosing the wrong 'tracks' to study (currently doing ML and Cybersecurity as I genuinely am interested in those fields). I won't be graduating until next year but I'm forcing myself to be optimistic that the tech job market will be in a better place by then.
by Kednicma on 8/31/20, 12:23 PM
It's not exactly a great year for extrapolating trends about what people are doing with their time. I wonder how much of this is 2020-specific and not just due to the natural cycle of AI winters.
by Ericson2314 on 8/31/20, 2:21 PM
Finally! Big companies need to realize they must understand what what they are doing with technology to get any value of out it.
They've long resisted that, of course, but I'm pretty sure half the popular of deep learning was it leveled the playing field, making engineers as ignorant of the inner-workings of their creations as the middle managers.
May the middle-manager-fication of work, and acceptance of ignorance that goes with, fail.
-----
Then again, I do prefer it when many of those old moronic companies flounder, so maybe this is a bad thing that they're wising up.
by SomeoneFromCA on 8/31/20, 12:30 PM
Deep Learning has become mainstream. The place work at actually uses 2 unrelated products based on NN.
by poorman on 8/31/20, 5:54 PM
I imagine this correlates to the "blockchain" postings.
by not2b on 8/31/20, 5:44 PM
I would have expected a comparison to job postings in general: how do deep learning job postings compare to job postings for any kind of technical position?
by realradicalwash on 8/31/20, 8:15 PM
Meanwhile, the academic job market, certainly in my area, ie linguistics/computational linguistics, has collapsed, too. A colleague did a similar and equally nice analysis here: https://twitter.com/ruipchaves/status/1279075251025043457
It's tough atm.
by kfk on 8/31/20, 5:15 PM
Data science and ML In big companies are pulling resources away from the real value add activities like proper data integrity, blending sources, improving speed performance. Yes Business Intelligence is not cool anymore. Yes I also call my team “data analytics”. But let’s not forget the simple fact that “data driven” means we give people insights when and where they need them. Insights could be coming from an sql group by, ML, AI, watching the flying of birds, but they are still simply a data point for some human to make a decision. That means we need to produce the insight, being able to communicate it to people, have the the credibility for said people to actually listen to what we are saying. Focusing on how we put that data point together is irrelevant, focusing on hiring PHDs to do ML is most likely going to end in a failure because PHDs are not predictive of great analytical skills, experience and things like sql are much better predictors.
by andrewprock on 8/31/20, 5:37 PM
On the plus side, ML systems have become commoditized to the point that any reasonably skilled software engineer can do the integration. From there, it really comes down to understanding the product domain inside and out.
I have seen so many more projects derailed by a lack of domain knowledge than I have seen for lack of technical understanding in algorithms.
by dboreham on 8/31/20, 3:12 PM
There will always be Snake Oil salesmen and hence Snake Oil..
by spicyramen on 8/31/20, 7:51 PM
Every company of course is very different, but I have seen that companies understood that fro Deep Learning you need a Pytorch or TF expert or maybe some other framework and most of these experts already work in Google/Facebook or any other advanced companies (NVIDIA, Microsoft, Cruise, etc), hiring is very difficult and cost is high. Then you can start using regular SQL and/or AutoML to get some insights. For a large number of companies that's enough. When there is so much complexity, such as DL modeling there's little transparency and management want to understand things. After COViD time will tell, but my take is that only a few companies need DL.
by x87678r on 8/31/20, 4:19 PM
In general does anyone know if its a good time to look for a new dev job? I was really going to move this year, but it seems sensible to wait. Just sucks to see friends with RSUs going up in value so quickly.
by gdsdfe on 8/31/20, 5:52 PM
For most companies ML is just part of the long term strategy, with covid priorities have shifted from long term R&D to short term survival, so I don't see anything out of the ordinary here
by samfisher83 on 8/31/20, 1:50 PM
A lot of thee c folks aren't tech folks or even math folks. They want to try to use deep learning to do prediction or get some insight when something as simple as regression would have worked.
by tanilama on 8/31/20, 5:44 PM
Deep Learning has been so commoditized and compartmentize over the past 5 years, now I think average SDE with some basic understanding of it can do a reasonable job in application.
by camoverride on 9/1/20, 1:50 AM
I don't think anyone should freak out when they see a tweet like this: deep learning is just one particularly trendy part of ML, which is just one piece of data science, which is just one job title in the "working with data" career space. I think that most people with backgrounds or interests in DL are very well equipped to participate in the (ever more important) data science world.
by alpineidyll3 on 8/31/20, 4:48 PM
Booms imply crashes. Anyone who is surprised at this couldn't be smart enough to be a good machine learning engineer.
by code4tee on 8/31/20, 10:49 PM
No question ML is powerful and can do great things. Also no question a lot of companies where just throwing money at stuff for fear of being seen as behind in this space. When the going gets tough such vanity efforts are the first things to go.
Teams adding measurable value for their companies should be fine but others might not be.
by astrea on 8/31/20, 2:15 PM
In my industry (research), we still have a strong line of business. Some commercial clients have killed their contracts with us to save money during the COVID era, but government contracts are still going strong. In areas where there's a clear use case I think there is still work to go around.
by darepublic on 8/31/20, 6:15 PM
My belief in an AI breakthrough is so strong that I would invite another AI winter to try to play catch up
by ponker on 8/31/20, 3:00 PM
The graph means very little without a comparison line of “all programming jobs” and/or “all jobs.”
by Traubenfuchs on 9/1/20, 7:28 AM
Good riddance. The majority of it is snakeoil, relabeling and "smoke and mirrors". A lot of smart or lucky people made a lot of money, a lot of dumb people with power over money lost... probably insignificant amounts of it.
by emmap21 on 8/31/20, 8:37 PM
ML/DL is at the exploratory phase for most companies. I have no surprise when seeing this post. Nevertheless, this also open new opportunities in other domains and new kind of business based on data. I have no doubt.
by ISL on 8/31/20, 6:17 PM
Is there a LinkedIn tool that allows you to make similar trend plots as shown in the Twitter thread, or has the author been archiving the data over time?
by rch on 8/31/20, 11:22 PM
Unless you're doing ML/DL/etc research then what you're really doing is engineering, like always.
by make3 on 8/31/20, 12:52 PM
the fact that he doesn't allow people to answer his tweets making data-less claims like this is really a problem
by hankchinaski on 8/31/20, 6:53 PM
covid has certainly sped up the transition to the "plateau" state in the ML/DL/AI hype cycle
by dgellow on 8/31/20, 6:18 PM
Is that a worldwide trend, or is it based on US data? That's not clearly stated in the tweet.
by MattGaiser on 8/31/20, 11:47 PM
How does that compare to job postings overall? Those would have fallen off a cliff as well.
by phre4k on 8/31/20, 11:10 PM
If you ever talked to one of the self proclaimed 'AI experts,' you know why.
by magwa101 on 8/31/20, 4:40 PM
Sufficient DL frameworks are now in the cloud and it is mostly an engineering problem.
by SrslyJosh on 8/31/20, 9:42 PM
I guess nobody's model... puts on sunglasses ...predicted this event.
by pts_ on 9/1/20, 2:26 AM
I have seen ML and big data crowd out remote openings though.
by arthurcolle on 8/31/20, 4:59 PM
Why was this headline changed?
by booleanbetrayal on 9/1/20, 1:06 AM
I believe this to be an obvious that the Singularity has already occurred.
by recursivedoubts on 8/31/20, 1:26 PM
memento mori: https://en.wikipedia.org/wiki/AI_winter
by rahimiali on 8/31/20, 9:23 PM
Citation needed.
by eanzenberg on 8/31/20, 4:27 PM
This needs to be normalized to “job posting collapse in the past 6 months” unless you expect DL jobs to grow while everything shrinks? I’m somewhat surprised by the analysis from someone’s who’s “data driven.” I mean, he even says so as much in the twitter thread:
“To be clear, I think this is an economic recession indicator, not the start of a new AI winter.”
So, looks like he discovered an economic recession.
by m0zg on 8/31/20, 9:11 PM
Out of curiosity: are there job postings that did not "collapse" over the past six months?
by bitxbit on 8/31/20, 6:38 PM
And yet data center spend has gone through the roof. Why?