How to Evaluate a Development Agency: Checklist for Founders

4 rescued projects in 18 months. That's how many times founders have come to us after another agency failed to deliver. One had spent EUR 80k on a half-built platform with no documentation. Another waited 8 months for an MVP that was supposed to take 3. A third received code so poorly architected that rebuilding from scratch was cheaper than fixing it.

Every single one of these founders had evaluated their original agency. They checked portfolios, read reviews, had discovery calls. They still chose wrong. Not because they were careless, but because they asked the wrong questions.

Here are the 10 questions you should ask, the 4 red flags to watch for, and the evaluation framework we'd use if we were hiring an agency ourselves.

Key Takeaways > - Portfolios and testimonials are curated marketing. They tell you what the agency wants you to see, not what you need to know. > - The most revealing questions are about process, failure, and handoff - not about technology or design. > - A great agency should make you comfortable enough to leave. If they create dependency, that's a strategy, not a feature.

The 10 Questions

1. "Can I talk to the person who will actually write my code?"

Most agencies put senior people on sales calls and junior people on your project. The articulate technical lead who impressed you in the pitch meeting might not touch your codebase.

Ask to speak directly with the engineer(s) assigned to your project. Not the account manager. Not the CTO who does pitches. The person who will write your authentication system and design your database schema.

At RalphNex, the people on the sales call are the people building your product. There's no handoff because there's no sales team. This isn't a flex - it's a structural advantage of small agencies. At a 50-person agency, it's reasonable to ask who specifically will work on your project and what percentage of their time is allocated to you.

2. "What happens when my project is done?"

This question reveals more about an agency than any portfolio review. Listen for:

- Good answer: "We hand off complete documentation, give your team a walkthrough, provide 30 days of support, and you own everything." This means they're confident in their work and don't need you to stick around. - Bad answer: "We offer ongoing maintenance and support packages." If this is the ONLY answer with no mention of documentation or handoff, they're building dependency. Maintenance revenue is how some agencies make most of their profit.

Every project ends. The question is whether it ends with you capable of continuing independently or chained to the agency for every bug fix.

3. "Show me a project that went wrong. What happened?"

Any agency that claims a perfect track record is lying or hasn't done enough work to have failed yet. Both are problems.

You want an agency that can describe a specific failure, explain what caused it, and tell you what they changed as a result. "We underestimated the complexity of a payment integration and had to extend the timeline by 3 weeks. Now we scope payment work separately with a dedicated buffer" is a great answer. It shows self-awareness and process improvement.

"All our projects have been successful" is the reddest of red flags.

4. "What's your process when the client wants to change scope mid-project?"

Scope changes happen on every project. The question is how the agency handles them.

- Good answer: A defined change request process. The change gets scoped, priced, and approved before work begins. The current sprint isn't disrupted. - Acceptable answer: "We use the project buffer for small changes and formally re-scope larger ones." - Bad answer: "We're flexible and accommodate changes as they come." This sounds nice. It means there's no process. Changes will creep in, timelines will slip, and nobody will know who's accountable.

5. "Can I see your code from a previous project?"

Not the live product. The actual code. Open a GitHub repository and look at it together. You don't need to understand every line. Look for:

- Consistent formatting. Sloppy formatting suggests no code review process. - Test files. If there are no tests, there's no quality assurance beyond manual checking. - README and documentation. If the repo has no setup instructions, imagine inheriting that codebase. - Commit messages. "fix stuff" and "WIP" everywhere means an undisciplined process. Clear commit messages mean clear thinking.

If the agency won't show you code from any project (even with client names redacted), that's concerning. We show anonymized code examples during our scoping calls because we're proud of the work, not hiding from it.

6. "What technology would you use for my project, and why not the alternatives?"

The "why not" part is critical. Any agency can tell you they'd use React and PostgreSQL. Fewer can explain why they wouldn't use Vue and MongoDB for your specific use case.

Good technologists have opinions and can defend them with tradeoffs. "We chose PostgreSQL over MongoDB because your data has strong relational patterns, and PostgreSQL's JSONB columns give us document flexibility when we need it" shows real technical thinking.

"We use React because it's the industry standard" shows a team that follows trends instead of evaluating options.

Here's the contrarian take: the best technology choice is often the boring one. If an agency is pitching you the latest framework released 6 months ago, they're optimizing for their own learning, not your product's stability. Proven, boring technology ships faster and breaks less.

7. "What does your sprint demo look like?"

You want to see working software regularly. Not Figma mockups. Not slide presentations. Working software running on a staging URL that you can click through yourself.

Ask how often you'll see the product during development. Every 2 weeks is good. Monthly is acceptable for smaller projects. "At delivery" is unacceptable. If you don't see the product until it's done, you have no opportunity to course-correct.

We demo on staging every 2 weeks. The client clicks through the features. If something's wrong, we catch it in the current sprint instead of discovering it 3 months later.

8. "Who owns the intellectual property?"

This should be obvious, but it's not always clear. You should own 100% of the IP for code written specifically for your project.

Watch for:

- Proprietary frameworks. Some agencies build your product on their internal framework. You own the product but not the framework, which means you need them (or their permission) to maintain it. - Shared libraries. If the agency uses open-source libraries, that's fine. If they use proprietary shared libraries across clients, your product has a dependency on code you don't control. - License restrictions. The contract should include a full IP assignment clause. No licenses, no restrictions, no "we retain the right to reuse."

Our contracts include complete IP assignment. Every line of code we write for you is yours. We use open-source tools and frameworks, never proprietary agency tooling.

9. "What's included in the price, and what's extra?"

Get a written list of what's included and what's not. Specifically ask about:

- Hosting setup (included or extra?) - Third-party service configuration (Stripe, SendGrid, etc.) - Documentation (what format, what depth?) - Post-launch bug fixes (for how long?) - App store submission (if applicable) - Content migration (if applicable) - Training/walkthrough sessions

We publish our inclusions and exclusions for every service tier. No surprises. The EUR 60k SaaS build includes everything from design to documentation to 30 days of post-launch support. Hosting costs and third-party subscriptions are yours.

10. "Can you give me 3 client references from the past 12 months?"

Not testimonials on the website. Real people you can call or email. Recent references matter because agencies change over time. A great 2023 project doesn't mean great work in 2026 if the team has turned over.

When you talk to references, ask:

- "Was the project delivered on time and on budget?" - "How was communication when problems arose?" - "Would you hire them again?" - "What's one thing you wish you'd known before starting?"

The last question is the most revealing. It surfaces the real experience, not the polished story.

The 4 Red Flags

These are the warning signs that should make you walk away, regardless of how good the portfolio looks.

Red Flag 1: No fixed pricing option

"We can't give a fixed price until we start" means one of two things: they can't scope accurately (inexperience), or they don't want to commit to a number (intentional vagueness). Either way, you're signing up for an unknown final cost.

Every reputable agency should be able to give a fixed-price or capped-price option after a scoping session. The scope might need adjustment, but the absence of any fixed pricing model is a warning.

Red Flag 2: The senior-to-junior bait-and-switch

Seniors on the pitch. Juniors on the project. This is the most common agency play and the hardest to detect before it happens. Mitigate by asking question #1 above and getting the specific team members written into the contract.

If the contract doesn't name the team members who will work on your project, assume they'll assign whoever is available - which usually means their cheapest resource.

Red Flag 3: No documentation in the deliverables

If documentation isn't explicitly listed as a deliverable in the proposal, you won't get documentation. "We document as we go" without specific deliverables means you'll get a README that says "run npm install."

Ask for the documentation template or a sample from a previous project. If they can't produce one, they don't have a documentation process.

Red Flag 4: They can't explain their failures

Every experienced agency has failed. Scope overruns, missed deadlines, unhappy clients - it happens. An agency that admits these experiences and explains what they learned is trustworthy. An agency that claims perfection is either lying, inexperienced, or both.

We've had projects go over timeline (never over budget, because fixed-scope). We've had clients who were difficult to communicate with. We've made technical decisions we'd reverse in hindsight. These experiences made our process better. Pretending they didn't happen would make us worse.

The Evaluation Scorecard

Rate each area 1-5 after your evaluation calls. Any score below 3 in any category should give you pause. A total below 30 means keep looking.

| Category | What to evaluate | Score (1-5) | |---|---|---| | Technical depth | Can they explain tradeoffs, not just choices? | | | Process maturity | Is there a defined process for sprints, changes, and handoff? | | | Transparency | Will they show code, name team members, discuss failures? | | | Communication | Are they responsive, clear, and direct? | | | Documentation | Is documentation a defined deliverable with templates? | | | Pricing clarity | Can they give a fixed or capped price with clear inclusions? | | | References | Can they provide recent, contactable references? | | | IP ownership | Is full IP assignment in the contract? | | | Cultural fit | Do they communicate the way you want to be communicated with? | | | Gut check | After all the evaluation, do you trust them? | |

The gut check matters. You'll work with this team for months. If something feels off during the evaluation, it will be 10x worse during a stressful sprint.

Frequently Asked Questions

How many agencies should I evaluate before choosing one?

Three is enough. Evaluate fewer and you don't have comparison data. Evaluate more and you're procrastinating. Get proposals from 3 agencies, run them through the 10 questions and scorecard, and decide within 2 weeks. The decision itself shouldn't take longer than the evaluation.

Should I choose the cheapest option?

Almost never. The cheapest agency quote is cheapest for a reason - junior developers, no documentation, cut corners on testing. We've rebuilt 4 projects from cheap agencies. Every rebuild cost more than the original project. Choose the best value (quality per euro), not the lowest price.

What if the agency I like doesn't do fixed pricing?

If they do time-and-materials only, negotiate a price cap. "We estimate 600 hours at EUR 100/hour (EUR 60k), and the total will not exceed EUR 72k regardless of actual hours." This gives the agency flexibility while protecting your budget. If they won't agree to any cap, they're not confident in their estimation, and you shouldn't be either.

Is it worth paying for a technical advisor to help evaluate agencies?

If you're non-technical, absolutely. A fractional CTO or senior freelance developer can join your evaluation calls, review code samples, and assess technical depth in ways you can't. Budget EUR 1k-2k for 5-10 hours of advisory time. It's the cheapest insurance you can buy against a bad agency choice that costs EUR 50k+ to fix.

*Looking for an agency that passes its own checklist? Book a 30-minute call and ask us all 10 questions. We'll answer every one. Or reach out through our contact page to start the conversation.*