Why I Don't Ban AI in Technical Interviews!

See my entire hiring process

The best developers use all the available tools to build the best applications as fast as possible. So, using AI day to day is part of the job. We use it to solve problems. We use it to complete code snippets, and testers use AI to dream up edge cases or user predictability.

AI Gernerated image using DALL-E

It's all around us and will become increasingly ingrained into what we do. So why would I remove elements of the job when interviewing candidates? I want to test them on the full picture, how they complete a real problem with all the cards on the table. In a remote interview setting, even if I wanted to ban AI, how would I be able to tell they were being truthful?

So, in 2023, when it came to hiring developers and testers for a new team I was building, I needed to hire 5 React developers with next.js and Typescript experience.

I sat down and reviewed my options. HR provided me with a Code testing tool where I could choose a topic, some questions, and varying levels and expanses of tests.

Then, I thought this would take a lot of time from the process and remove the team culture and engagement I wanted to build. I wanted to start as I meant to go on, and I knew I wanted a number of things to happen.

• it needs to be interactive

• I need to understand the candidate's reliance on AI

• and the need to be able to solve our real-world problems

So this is what I strated with, and it created the foundations for all my future interviews at all levels and was recognised by HR as the North Star, the gold standard in the hiring process.

The Technical Interviewing Process

Hiring great developers requires balancing assessing real-world problem-solving skills, integrating AI into workflows, and keeping the process engaging for both candidates and the hiring team. A well-structured technical interviewing process ensures that candidates can demonstrate their expertise to reflect the challenges they will face on the job while also allowing them to showcase how they leverage AI as part of their workflow.

Defining the Role and Expectations

Before starting the hiring process, it's essential to clearly define success for the role. Beyond technical skills, a strong developer should possess problem-solving abilities, adaptability, and the ability to collaborate effectively. AI familiarity is now an essential skill, so the ability to use tools like GitHub Copilot and ChatGPT while maintaining critical thinking is a key aspect of evaluation.

Finding and Screening Candidates

Sourcing candidates should focus on platforms where developers actively engage, such as GitHub, Stack Overflow, and AI-driven coding communities. Candidates with open-source contributions, technical blogs, or active participation in discussions around emerging technologies should be given special attention.

Pre-screening questions should go beyond technical proficiency and focus on real-world experience. Instead of just asking about skills, assess how candidates approach challenges with questions like:

• "Describe a complex problem you've solved and how you approached it."

• "How do you integrate AI into your daily development workflow?"

  
  

These questions provide deeper insights into a candidate's mindset and approach to problem-solving.

Stage 1: Phone or Video Screening

The first stage of the interview process typically starts with a phone or video screening. This is an opportunity to assess both technical thought processes and communication skills.

A helpful approach is to present a simple technical challenge and ask the candidate to discuss their solution. AI usage should be allowed, but the focus should be on how the candidate interprets AI-generated suggestions, validates them, and applies their reasoning.

This stage is also a chance to discuss past experiences, teamwork, and learning strategies. Asking candidates how they have handled challenges in previous roles provides valuable insight into their approach to problem-solving.

Stage 2: Hands-On Technical Challenge

Rather than relying on generic coding tests, the best approach is to give candidates a problem derived from the company's actual codebase, with necessary modifications for confidentiality.

For example, optimising a slow-running API request in a React and Node.js application provides a realistic scenario that tests problem-solving ability in a practical setting.

AI usage should be encouraged, but candidates should be required to explain:

• Why did they make specific choices

• How they validated AI-generated solutions

• What improvements did they make beyond what the AI suggested

  
  

The most effective way to run this challenge is through a collaborative coding session, in which a team member works alongside the candidate, replicating real-world problem-solving dynamics.

Stage 3: System Design Discussion

Following the coding challenge, a system design discussion provides deeper insights into a candidate's architectural thinking. Rather than focusing on coding syntax, this stage presents open-ended business problems such as:

• "Design a real-time analytics dashboard for a telecom company handling millions of data points per second."

  
  

The discussion should cover data modelling, caching strategies, microservices, and scalability considerations. Candidates should be encouraged to draw diagrams, explain trade-offs, and justify their decisions, offering a glimpse into their ability to navigate complex engineering challenges.

Stage 4: Culture Fit and Leadership Assessment

The final interview stage focuses on culture fit and leadership qualities. Candidates engage with engineers, product managers, and leadership to evaluate their ability to:

• Lead technical discussions

• Handle evolving project requirements

• Communicate effectively across teams

  
  

This stage helps determine how well the candidate aligns with the company's values and long-term vision while assessing their ability to contribute beyond just writing code.

Evaluating Candidates and Making Decisions

Technical proficiency remains critical, but using AI effectively while maintaining independent problem-solving skills is equally important. Collaboration, adaptability, and structured thinking should carry significant weight in decision-making.

Candidates should be assessed on:

• Their ability to think critically

• Their approach to proposing multiple solutions

• Their clarity in communicating technical reasoning

  
  

An intense interview process does not end with selection. Providing clear feedback to candidates, whether they move forward or not, helps build a reputation as a company that values talent and growth. Internally, post-interview debriefs ensure alignment among the hiring team and provide insights into refining future interviews.

Building a Future-Proof Interview Process

Hiring teams can ensure that the interview process remains engaging and effective by structuring it around real-world challenges, AI-enhanced workflows, and collaborative problem-solving. Developers thrive when given opportunities to showcase their skills meaningfully, and a well-designed interview process helps companies attract and retain the best talent.

The Interviewing Process for Managers and Senior Engineering Managers

Hiring engineering leaders requires a distinct approach compared to hiring individual contributors. Beyond technical expertise, engineering managers and senior managers must demonstrate strategic thinking, leadership, stakeholder communication, and the ability to drive organisational and technical change. A well-structured interview process should assess their ability to navigate complex projects, lead teams effectively, and align technology with business goals.

Defining the Role and Expectations

A strong engineering leader balances technical depth with the ability to influence, coach, and drive alignment across teams. They should be able to:

• Lead and mentor teams to improve productivity, collaboration, and problem-solving.

• Communicate effectively with stakeholders, translating business needs into technical execution.

• Drive decision-making around technology, architecture, and engineering strategy.

• Manage dependencies, risks, and escalations while ensuring technical excellence.

• Adapt to AI-driven changes in engineering and incorporate automation, data-driven insights, and emerging technologies into their leadership approach.

  
  

Finding and Screening Candidates

The hiring process should focus on sourcing candidates with strong technical foundations and leadership experience in complex, large-scale environments. The best candidates often contribute to engineering communities, write about technology leadership, or actively engage in technical discussions on platforms like LinkedIn, GitHub, and industry conferences.

During pre-screening, structured questions should assess both technical and leadership capabilities. Some key areas to explore include:

• Experience leading engineering teams and driving impact at scale.

• Decision-making in ambiguous, high-stakes environments.

• Communication skills when dealing with executives, engineers, and cross-functional teams.

• Familiarity with AI and automation in engineering management.

• Approaches to team performance measurement and improvement.

  
  

Candidates should be asked questions like:

• "Tell us about a time you had to make a high-impact technical or organisational decision. What trade-offs did you consider?"

• "Describe how you've used AI, data, or automation to improve engineering team efficiency or software quality."

• "How do you balance business priorities with engineering excellence when making trade-offs?"

  
  

These discussions reveal how candidates approach leadership, problem-solving, and technical execution at scale.

Stage 1: Initial Interview – Leadership and Communication Assessment

The first stage of the interview should focus on leadership philosophy, communication style, and stakeholder management. Strong engineering leaders must be able to translate technical challenges into business terms and align diverse teams around common goals.

A scenario-based discussion is often the most effective way to evaluate these skills. Some potential questions include:

• "You've inherited a struggling engineering team. How do you assess the issues and drive improvement?"

• "You need to convince leadership to invest in a new technology initiative. How would you structure your argument?"

• "A critical production issue has caused a major outage. How do you manage the response while keeping leadership and customers informed?"

  
  

This conversation provides insight into how candidates handle complexity, drive alignment, and lead under pressure.

Stage 2: Strategic and Technical Problem-Solving

Engineering managers and senior leaders must be able to design, guide, and refine technical strategies while aligning them with business objectives.

This stage involves real-world engineering and leadership challenges, which may include:

• Stakeholder Communication and Decision-Making: Candidates are given a scenario where they need to align teams around a major technical decision. They must outline how they gather input, communicate decisions, and handle resistance.

• Scaling Engineering Organisations: Candidates discuss how they have scaled teams, optimised engineering processes, and handled rapid growth or reorganisation.

• AI and Automation Strategy: Candidates are asked how they have integrated AI, automation, or data-driven decision-making into engineering leadership.

• Technical Risk Management: A hypothetical crisis (such as a security breach or significant system failure) is presented, and candidates must walk through how they would assess and mitigate risks.

  
  

This discussion should focus on practical applications rather than theoretical answers. Candidates should be expected to think on their feet, demonstrate strategic vision, and communicate clearly.

Stage 3: Culture, Execution, and People Leadership

The final stage of the interview assesses how well the candidate aligns with company culture, manages execution, and builds high-performing teams.

Key discussion areas include:

• Building and Retaining High-Performing Teams – How they recruit, mentor, and create an inclusive and productive team environment.

• Managing Conflict and Performance Issues – Their approach to handling underperformance or team misalignment.

• Driving Execution in Fast-Paced Environments – How they prioritise initiatives, deliver results, and balance innovation with stability.

• Adapting to Changing Technologies – How they continuously learn and help teams adapt to AI-driven engineering and evolving best practices.

  
  

Scenarios should be open-ended, allowing candidates to share real examples from their experience. Questions might include:

• "Tell us about a time you had to rebuild trust in an engineering organisation after a major failure."

• "How do you measure engineering success beyond just delivery timelines?"

• "What's your approach to balancing technical debt and innovation?"

  
  

This stage ensures that candidates align with the company's vision while demonstrating leadership skills that foster team growth and execution excellence.

Final Evaluation and Decision Making

Selecting a senior engineering leader requires balancing technical expertise, leadership acumen, and business impact. Key evaluation criteria include:

• Strategic Thinking – Do they make thoughtful trade-offs that align engineering and business needs?

• Communication Skills – Can they clearly articulate complex concepts to different stakeholders?

• Decision-Making Under Uncertainty – How well do they handle ambiguity, risk, and changing priorities?

• AI and Automation Readiness – Do they leverage modern tools to improve engineering efficiency?

• People and Culture Leadership – Can they build, retain, and inspire high-performing teams?

  
  

A firm hiring process does not end with selection.

Providing detailed feedback to candidates, whether or not they are hired, reinforces a reputation as an organisation that values talent and development. Internal debriefs among the hiring team also help refine future hiring strategies.

Building an Effective Leadership Hiring Process

Hiring engineering managers and senior leaders is about more than just technical proficiency. The best leaders drive impact through vision, execution, and team enablement. By structuring the interview process around real-world challenges, AI integration, and strategic decision-making, companies can ensure they attract and select leaders who will drive innovation, foster strong teams, and align technology with business goals.

The Success of Hiring

The Real-World Approach to Hiring – Ditch the Lazy Tech Tests

Hiring should never involve arbitrary IQ tests or hypothetical exercises that are irrelevant to the job. Predicting the size of a pointless object in a tech test is a waste of time. Instead, the focus must be on real-world problem-solving—evaluating candidates in the context of the actual work they'll be doing. A firm hiring process is an investment in your team's future, ensuring that new hires are technically skilled and aligned with your organisation's challenges, expectations, and culture.

The Four-Step Hiring Process That Works

1. Understand Your Problem Before thinking about hiring, define the problem you need to solve. Clarity on your challenges is essential, whether scaling an engineering team, optimising a legacy system, or driving AI adoption. Hiring without a clear problem statement leads to mismatches and wasted time.

2. Show the Candidate the Problem Instead of testing for abstract problem-solving skills, bring candidates into the real-world issues your company faces. Present a modified real problem from your codebase, product, or team challenges. This allows them to showcase how they would approach, diagnose, and solve something they'd be working on.

3. Score How They Fix Your Problem Candidates should be evaluated on their thought process, collaboration, decision-making, and execution. AI can be part of their workflow, but they must show how they validate and refine AI-generated solutions. Score their approach based on well-defined, realistic success criteria, not just whether they arrive at a "correct" answer.

4. Compare Fairly Using a Predefined Framework A structured, fair hiring process ensures the same criteria judge every candidate. Subjective gut feelings lead to inconsistent hiring. Instead, use a predefined evaluation framework to compare how each prospect tackled the challenge, communicated their solution, and fit the role's expectations.

Hiring Is Your Number One Priority

A hiring manager's most important job is hiring. Doing this right now will save you work later. Rushing or cutting corners leads to bad hires, poor team dynamics, and productivity losses. A strong hiring process is the foundation of an autonomous, high-performing team—one where talented individuals are engaged, aligned, and fully aware of what they're signing up for.

Look at The Apprentice—candidates are tested in a rigorous hiring process each week until only one remains. The same principle applies to building great teams. Only those who can handle the job, thrive in the environment, and align with the company's expectations should make it through.

This is not about finding the "smartest" person in the room. It's about finding the right person for the job—someone who understands the work embraces the challenge, and commits to the role with full awareness of what's expected. Dedication to the process will reward you with better hires, stronger teams, and less firefighting down the line.

Documenting the Process – Preparation Is Everything

A great hiring process isn't just about running solid interviews; it's about documenting and refining the process to ensure consistency, fairness, and long-term success. Preparation is key. Without a structured approach, hiring becomes chaotic, biased, and unreliable.

Every step should be documented, from job role definitions to interview frameworks and evaluation criteria. This ensures that every candidate is assessed on the same standards and allows the team to reflect on what works and what doesn't. A well-documented hiring process also helps scale hiring across multiple teams, ensuring quality doesn't drop as the company grows.

Preparation means having:

• A clearly defined role with expectations set upfront.

• Real-world problems are prepared in advance for candidates to work through.

• A scoring framework to objectively compare candidates.

• An AI and automation strategy for hiring, ensuring efficiency without losing human judgment.

  
  

The Importance of Feedback – Closing the Loop

Hiring doesn't end when a candidate is selected or rejected. Clear and structured feedback is essential for the candidate and the hiring team.

For Candidates: Whether they get the job or not, feedback helps them understand what they did well and where they can improve. This strengthens your company's reputation and ensures positive engagement in the tech community. Even candidates who aren't selected should walk away feeling they had a fair shot and learned something from the process.

For the Hiring Team: Post-interview debriefs should be part of the hiring process. Discussing what worked and what didn't helps refine the approach over time. If multiple candidates struggled with the same aspect, it might indicate a flaw in the interview process, not just in the candidates.

The Positive Outcome – A Self-Sustaining, High-Performing Team

When hiring is done right, the rewards are massive. A structured, fair, and engaging hiring process leads to teams that are:

• Self-sufficient and autonomous – hiring the right people means less micromanagement.

• Aligned with expectations – no surprises when they start the job, leading to higher retention.

• Engaged and motivated – they know what they signed up for and are ready to contribute from day one.

• Future-proof – an AI-ready team that understands how to integrate modern tools while still thinking critically.

  
  

Hiring is not just about filling a role today—it's about building the foundation for long-term success. Companies that invest in structured, real-world hiring processes today will save themselves a world of headaches later. The best teams don't happen by chance—they're built through deliberate, disciplined, and well-documented hiring practices.

Thomas Archer