Skip to main content

How to Vet a Software Development Company: An Evidence-Based Buyer’s Checklist

Six evidence-based steps to assess quality, contracts, security, and team structure

How to Vet a Software Development Company: An Evidence-Based Buyer’s Checklist
Topic Devs
Updated
Author Samuel Jim
Read Time 11 min

Quick Take

The strongest signal in vendor selection is paperwork, not presentation. Before you meet a shortlist, decide which quality dimensions matter and how you will measure them, choose a contract structure that prices your actual scope uncertainty, and put acceptance and warranty terms in writing. Then treat security and team structure as their own scored questions. A vendor who can answer all four in writing is a different kind of risk than one who can only show you a portfolio.

Most buyers pick a software development company the way they pick a restaurant: they look at the photos, read a few reviews, and trust the vibe. That works until the project slips a quarter, the invoice grows, and nobody can agree on whether the delivered product is actually finished.

The fix is unglamorous. You vet a vendor against criteria you wrote down before you met them: quality characteristics, contract structure, quality assurance and acceptance terms, security risk management, and team structure. None of that requires a procurement department. It requires a checklist and the discipline to use it.

This guide covers software and ICT product acquisition: custom builds, platform work, and system delivery. The contract references come from US federal acquisition rules, which are the most thoroughly documented public example of structured buying. Treat them as a model for how to think about risk, not as commercial law in your jurisdiction. For anything binding, get your own legal review.

Info

If you are evaluating a software development company for a custom build, run the checks below against their written proposal and contract draft, not against a sales deck.

The Vetting Checklist: Six Steps From Shortlist to Signature

Work through these in order. Each step produces a document you can point to later, and each one narrows what the next step should say. Skipping ahead, such as negotiating price before you have acceptance criteria, is how buyers end up arguing about “done” in month nine.

Step 1: Write Down What “Good” Means Before You Talk to Anyone

Quality criteria are the foundation of every later check. Without them, acceptance criteria are arbitrary and contract disputes have no referee.

  1. Adopt a quality model as your requirements skeleton. ISO/IEC 25010:2023 defines a product quality model for ICT and software products built from nine characteristics, each subdivided into subcharacteristics. The standard explicitly supports eliciting and defining requirements, validating that requirements are comprehensive, setting testing objectives, and identifying acceptance criteria. Use it as a checklist of quality dimensions you might otherwise forget, including performance efficiency, reliability, security, maintainability, and usability.
  2. Turn each relevant characteristic into a testable statement. “Should be fast” is not a criterion. A statement that names the measure, the threshold, and the load condition is. For every characteristic that matters to your product, write the measure and the threshold. If a characteristic does not matter for this build, say so explicitly and note why.
  3. Send the criteria with the request for proposal. Ask each vendor to respond to your criteria line by line. Vendors who ignore the structure or answer with marketing language have told you something useful about how they will handle your requirements later.

Verify the result

  • Every quality characteristic you care about has a written measure and threshold.
  • Characteristics you deliberately excluded are documented with a reason.
  • At least one vendor has returned a line-by-line response rather than a brochure.

Step 2: Match the Contract Type to How Certain Your Scope Really Is

Contract type is a risk-allocation decision, and most buyers make it by accident. FAR 16.101 lays out the logic cleanly: contract types vary according to the degree and timing of the contractor’s responsibility for performance costs, and the amount and nature of the profit incentive. They group into two broad categories, fixed-price and cost-reimbursement, and range from firm-fixed-price, where the contractor carries full responsibility for performance costs and the resulting profit or loss, to cost-plus-fixed-fee, where the contractor has minimal cost responsibility and the fee is fixed. Incentive contracts sit in between, tailored to the uncertainties involved.

Translated for a commercial buyer: the more certain your scope, the more cost risk you can push onto the vendor. The less certain your scope, the more you pay for that transfer, either in a padded fixed price or in change orders.

Contract type vs. scope certainty
Your scope situationContract type to considerWho carries cost riskWhat to watch
Requirements are defined, stable, and testableFirm-fixed-priceVendor bears performance costs and resulting profit or lossChange-order pricing; make the change mechanism explicit
Some uncertainty, but outcomes are measurableFixed-price incentive or similar incentive structureShared, with incentives tied to cost, performance, or deliveryWhether the incentive targets are meaningful or cosmetic
High uncertainty — discovery work, novel product, shifting requirementsCost-reimbursement or time-and-materialsBuyer bears more of the cost riskCeilings, reporting cadence, and stop-work triggers

FAR Part 16 lists the full menu: fixed-price variants, cost-reimbursement contracts, incentive contracts, indefinite-delivery contracts, and time-and-materials, labor-hour, and letter contracts. You do not need all of them. You need to know which one your scope justifies, and to notice when a vendor pushes a fixed price onto a project nobody can define yet. That combination usually means the price is a placeholder and the change orders are the real business model.

Step 3: Lock Quality Assurance, Acceptance, and Warranty Terms

This is the step buyers underrate most. A glossy portfolio is a marketing artifact; a written acceptance criterion is an enforcement mechanism. FAR Part 46 structures the whole apparatus: contract quality requirements, contractor inspection obligations, government contract quality assurance at source and destination, acceptance responsibilities, nonconforming supplies or services, and warranties, including criteria for their use and their terms and conditions.

You are not buying federal supplies, but the anatomy transfers directly. Your contract should answer four questions in writing: what quality requirements apply, who inspects and when, who formally accepts the work and against what criteria, and what remedy exists when the delivered product fails.

Contract terms to confirm before signature

  • Acceptance criteria are written, testable, and traceable to your Step 1 quality criteria.
  • Inspection responsibilities are assigned, covering who tests, at which stage, and with what evidence.
  • A formal acceptance event exists, with a defined sign-off authority on your side.
  • Nonconforming deliverables trigger a defined remedy, not an informal conversation.
  • Warranty scope, duration, and exclusions are stated in the contract body.

Step 4: Assess Security Risk Management as Its Own Axis

Security is not a line item you bolt onto the technical questionnaire. It is a separate evaluation dimension, and vendors who treat it as a compliance checkbox tend to treat it the same way during delivery.

The NIST Cybersecurity Framework exists to help organizations better understand and improve their management of cybersecurity risk. That framing is useful in a vetting conversation: you are not asking whether a vendor is “secure,” because nobody can answer that, you are asking how they identify, assess, and manage risk across the life of your product. Ask for their risk management process, how it maps to your threat model, who owns security decisions, and how vulnerabilities are triaged and disclosed. Vague answers here are a genuine signal.

If the engagement involves cloud migration or infrastructure change, the failure patterns are well documented. See our breakdown of why cloud migrations fail for the risk parallels worth raising in the same conversation.

Step 5: Verify Team Structure and Delivery Process

Team claims are easy to make and easy to check. If a vendor says they run Scrum, hold them to what the framework actually specifies. The 2020 Scrum Guide states that the Scrum Team is small enough to remain nimble and large enough to complete significant work within a Sprint, typically 10 or fewer people, and defines the roles of Product Owner, Scrum Master, and Developers, along with the events and artifacts.

That gives you concrete questions: Who is the Product Owner on our engagement, and do they have real authority to order the backlog? Who is the Scrum Master, and is that a full role or a hat someone wears on Fridays? How many people are on the team, and does the number match the framework the vendor claims to follow?

Warning

Team-size guidance is Scrum-specific. A vendor running Kanban, a stage-gate process, or a hybrid model is not bound by the Scrum Guide’s team composition. Judge them against the process they actually claim, and be suspicious of a vendor who cannot name their process at all.

Step 6: Confirm Life-Cycle and Acquisition Process Alignment

Ask how the vendor’s delivery process connects to your acquisition process: requirements handoff, change control, release management, transition to operations. ISO/IEC/IEEE 12207:2017 describes software life-cycle processes that can be applied during the acquisition of a system that contains software, either alone or alongside the system life-cycle standard ISO/IEC/IEEE 15288:2015. It is a useful map of the process territory a mature vendor should be able to speak to.

Danger

ISO/IEC/IEEE 12207:2017 is marked Withdrawn on the ISO catalogue. Use it as a conceptual reference for process coverage, and confirm with the vendor which current standard or internal process framework they actually follow. Do not cite a withdrawn edition as a compliance requirement in your contract.

For supplementary practice material, the SEI Digital Library hosts software acquisition and quality publications, including acquisition guidance and quality models for machine learning components. That reading is useful if your product involves ML and your vendor’s quality claims need sharper questions.

What to Watch For: Red Flags and How to Handle Them

The vendor answers your quality criteria with marketing language instead of measures

This is the most common failure mode. A vendor who cannot restate your criteria as testable commitments will not meet them under delivery pressure. Ask for a rewritten response with measures and thresholds. If the second attempt is still vague, that is your answer.

The price is fixed but the scope is not defined

A firm-fixed-price bid on an undefined scope is a placeholder. Expect change orders to carry the real margin. Either invest in defining scope first, or move to a contract structure that prices uncertainty openly.

“Acceptance” is described but never defined

If the contract says the client accepts the work without stating the criteria, the sign-off authority, or the remedy for nonconformance, you have no enforcement mechanism. Rewrite the clause before signature, because this is far cheaper than litigating it later.

Security is answered with a certification badge

A certificate proves an audit happened at a point in time. Ask instead how the vendor identifies and manages risk on your engagement, who owns security decisions, and how vulnerabilities are triaged. A certification plus a vague process answer is weaker than a clear process answer alone.

The team is described as “senior” with no named roles

Ask for named roles against the process the vendor claims. If they say Scrum, ask who the Product Owner and Scrum Master are and how large the team is. If they cannot answer structurally, the process claim is decorative.

The Consolidated Checklist

Run this before you sign

  • Quality criteria written, measured, and sent with the RFP.
  • Contract type matched to scope certainty, with the risk allocation understood on both sides.
  • Acceptance criteria, inspection responsibilities, nonconformance remedies, and warranty terms in the contract body.
  • Security risk management process documented, with a named owner.
  • Team roles and size verified against the process the vendor claims.
  • Life-cycle and acquisition process alignment confirmed, using current standards rather than withdrawn editions.

Vetting a software development company is a documentation exercise. The vendors worth hiring will not be annoyed by a structured checklist, because they will recognize it as how their better clients buy. The ones who push back on written acceptance criteria or get vague about risk allocation have told you exactly how the project will go.

Start with Step 1 this week. Write your quality criteria before your next vendor call, and you will walk into that meeting with more leverage than any reference check can give you.

Key Takeaways

  • Write quality criteria before you evaluate vendors, because they anchor every later check.
  • Match contract type to scope certainty; a fixed price on undefined scope is a placeholder.
  • Acceptance criteria and warranty terms are your enforcement mechanism, not paperwork.
  • Treat security as its own evaluation axis with a named owner and a documented process.
  • Verify team structure against the process the vendor actually claims.
Samuel Jim

About the Author

Samuel Jim

Samuel Jim Nnamdi is a senior software engineer. He has over 8 years of software engineering and cybersecurity expertise.

View all posts by Samuel Jim →
Comments

Be the First to Comment