Looking for new data science opportunities in this heated market? Before you accept that offer, here are some red flags to beware of. While these are from the perspective of data science, they would also apply to most tech roles.

Data: No data, or data that’s poorly organized and/or inaccessible. This can happen if you’re joining a startup that doesn’t have a critical mass of customers yet or if it’s a B2B company that relies on partners for data (e.g., ML solution providers for healthcare, fintech). Almost as bad, they might have data, but it’s stored in various formats with different schemas, making it difficult to use.

Without basic data infra in place, a data scientist would face an uphill battle trying to contribute value via analytics or machine learning. Most will likely be frustrated with the grind of data acquisition, organization, cataloging, and building pipelines.

A few questions to suss out this red flag:

• What data is being generated or collected by your systems?
• What are the key objects in your data, such as customers, items, or transactions, and what is the approximate number of new rows daily or monthly?
• As a new joiner, how would I access the data?

Roadmap: No/poor plan on how the team will deliver value to customers and the business. A few years ago, during an interview, I asked a director of data science about his roadmap for the next 12 months. He responded at length about how he was focusing on team growth, diversity, and upgrading the tech stack. (This wasn’t the answer I was looking for.) I then asked, more specifically, how the team would contribute to customers and the business. Unfortunately, he couldn’t answer anything coherent—that was a big red flag for me.

While data science isn’t the only one guilty of this, it’s especially easy for us to make this mistake. Engineering and product can point to concrete features that improve the customer experience and contribute to the bottom line. But data science can spend too much time on research without tangible results. While this is acceptable if you’re in an innovation team, typically, most companies don’t have the luxury of funding such a team. And when lean times come, teams that don’t contribute to the business are likely first to be cut. Also, if you’re a practitioner, it’s not healthy for your career and resume to have too many years go by without delivering customer or business results.

Questions to ask about the roadmap:

• How does the team deliver value to customers and the business?
• What is your roadmap and planned deliverables for the quarter? And the year?
• How does success look like for the team? What are the team’s KPIs?

Role: Misaligned expectations on the role and career progression guidelines. The term “data scientist” has become synonymous with “data analyst” in certain companies. In August 2017, the Reddit community pointed out that data scientists at Facebook (now Meta) were mostly doing data analyst work. Similarly, in April 2018, Lyft rebranded their data analysts as data scientists, explaining that they were losing data analytics candidates to competitors offering the data scientist role.

Since then, several companies have gone down the same path, to the extent where their data scientists mainly build data pipelines and dashboards. The point is, job title is often a poor signal of what the role involves, and accepting an offer based solely on the title can lead to misaligned expectations, job dissatisfaction, and eventual attrition. Thus, don’t make assumptions based on the title and clarify what the role entails. (I previously did a comparison of data, research, and applied scientist roles.)

If you can get access to the team’s promotion guidelines, consider whether it aligns with your aspirations. For example, if you prefer building simple, effective machine learning systems, guidelines that emphasize research and state-of-the-art might require you to do a separate promo project, thus delaying your progression. On the other hand, if your preference is research and publishing but the guidelines require a record of system design and engineering, you might find yourself stuck down the road.

Questions to ask about the role and progression:

• What should someone in this role deliver in the first 100 days? And the first year?
• Of the most effective people in this role, what key skills and behaviors do they have?
• How is success in this role defined and measured?
• What does someone have to demonstrate to progress to the next level?

Manager: Incompetent, lacks vision, abusive, etc. We each have different yardsticks on what makes a good boss. Similarly, what makes a manager terrible likely differs from person to person. Thus, it’s hard to give prescriptive advice on how to probe for this red flag. One way is to ask the hiring manager to share the contact details of 2 - 3 people who used to work with them, people who can speak freely without fear of direct repercussion. Alternatively, speak to the most tenured person on the team.

Questions to probe for managerial red flags:

• What are the strengths and weaknesses of the manager?
• Did you enjoy working with the manager? Why or why not?
• How did the manager support the team and help them grow?
• Would/did you enjoy hanging out with the manager in a casual setting?

Tooling: They use outdated or proprietary tools that are barely transferable to other roles. It takes considerable time and effort to learn new tools before we’re productive with them. Yet, our proficiency with tools is perishable—we lose touch if we stop using them for a prolonged period. Thus, it’s natural to have concerns about using outdated or proprietary tooling (specific to data science, the big “S”s), especially since they’re likely non-transferable to other roles. Also, using good tools makes the work more enjoyable.

Some questions on the tooling:

• What key tools does the team use in their day-to-day?
• What is your tech stack?

Org structure: The data science team rolls up to an unusual C-level. Previously, I interviewed with a scale-up that was planning to IPO (and did IPO at the end of 2021). There, the data science team reported directly to the CFO. With some probing, I got the sense that the DS team mostly worked on preparing investor reports and tracking financial metrics, with the overall goal of the IPO. This was a red flag for me.

While there isn’t a definitive C-level for the data team to report to, I’ve seen organizations where data team rolled up to CTO or CIO or VP Eng that seem to work well. If the org has a CDO, that would be ideal. I would be concerned if data team reported to the CFO or CMO.

Questions to ask on org structure:

• Who does the data science team roll up to?
• Who are the main stakeholders of the data science team?

Iteration speed: The team moves too fast/slow for your liking. I’ve been on teams where you can deploy a new model in minutes and run an AB test every two weeks, and I’ve been on teams where deployment meant several people nursing the pipeline overnight, with little to no AB testing throughout the year. The former allows rapid experimentation, learning, and growth, while the latter leads to friction, sluggishness, and stagnation.

That said, I know data scientists who have no qualms with spending several months on research and only validating their hypotheses after a year or two. You know yourself best; pick a pace that aligns with how fast you want to iterate.

Questions to get a sense of the team’s iteration rate:

• What was the last thing the team shipped? How long did that take?
• How many AB tests did the team conduct in the last month or quarter? What were the outcomes?

• • •

You might ask, how do I find time for all these questions, especially if I only have time for questions in the last 5 - 10 minutes of each interview?

First, you don’t have to ask all the questions—focus on what’s important to you. It’s hard to find a role that doesn’t have any red flags and has great compensation, work-life balance, etc.; so manage your expectations.

Second, you can ask for follow-up chats with the hiring manager, and another member of the team, where you can reverse interview them. There’s little to lose from this request and what you learn can either help you dodge a bullet or set you up for success in your new role.

Thanks to Yang Xinyi for reading drafts of this.

If you found this useful, please cite this write-up as:

Yan, Ziyou. (Feb 2022). Red Flags to Look Out for When Joining a Data Team. eugeneyan.com. https://eugeneyan.com/writing/red-flags/.

or

@article{yan2022redflag,
  title   = {Red Flags to Look Out for When Joining a Data Team},
  author  = {Yan, Ziyou},
  journal = {eugeneyan.com},
  year    = {2022},
  month   = {Feb},
  url     = {https://eugeneyan.com/writing/red-flags/}
}