Rainforest QA vs. hiring a QA engineer

Costs

Budget-friendly

Hiring: ❌ No

Hiring a QA engineer is expensive. ​

Publicly-available salary databases show that hiring an experienced QA engineer in the U.S. tends to cost north of $100K.

And that doesn’t include the additional costs of QA tooling and infrastructure, let alone the time, money, and energy costs of the hiring process.

If you don’t have the budget but still want to enjoy the benefits of a full-time hire, you could consider an offshore hire. You’ll save money, but could introduce productivity challenges stemming from your team working across distant time zones and potential language barriers.

Rainforest: ✅ Yes

Rainforest's annual plans start at a quarter of the cost of hiring an experienced QA engineer in the U.S. — even when your plan includes a dedicated Test Manager who handles 100 percent of test writing and maintenance for your team.

Low financial risk

Hiring: ❌ No

If a QA hire doesn’t work out, there’s no way to recoup the time, money, and effort you and your team put into recruiting, hiring, onboarding, training, managing, and salary. You’re worse off than before you decided to hire in the first place.

Rainforest: ✅ Yes

Rainforest offers a 60-day money-back guarantee any time we add a Test Manager to your team.

Before you sign up for a plan, we’ll agree on your measurable success criteria for your first 60 days with us. If we don’t meet these goals within 60 days, you can ask for a refund. We’ll give you your money back, no questions asked.

Includes tooling & infrastructure

Hiring: ❌ No

When you hire a QA engineer, you still need to provision and manage the tools and infrastructure they’ll use.

For example, you’ll need:

• Plugins to extend the functionality of your open-source testing framework (e.g., for visual regression testing, detailed test results, or test recording).
• Real or virtual machines on which to run your tests — whether you use your own machines or a third-party “test grid” like BrowserStack.
• Software to manage your test suite.

Rainforest: ✅ Yes

Rainforest includes all the tooling and infrastructure you need for the entire testing workflow.

Within the platform, you can:

• Create, edit, and organize your automated tests.
• Run tests hundreds of tests in parallel on our cloud of Windows and macOS virtual machines — no configuration required.
• Get detailed test results, including debugging steps, video recordings, and http and browser logs.

Velocity

Fast time to initial results

Hiring: ❌ No

Once you decide to add a QA engineer role to your team, finding and hiring good candidates is typically quite time-consuming, since it involves the common process of posting the job opening, screening, interviewing, negotiating, onboarding, and training.

Rainforest: ✅ Yes

Rainforest is designed to help you move fast. We can have your team in place within days of signing a deal. Usually, your Rainforest Test Manager can start creating automated test coverage on their first day, getting your smoke suite up and running within a week.

Fast test creation & maintenance

Hiring: ❌ No

Creating and maintaining tests in open source automation frameworks is notoriously painful and slow, especially for a software team trying to move fast — a person with the right technical skills has to dig around in and manipulate code, often bottlenecking the release pipeline.

And, at least so far, AI doesn't appear to help speed things up.

The more complex your app becomes, the more someone needs to investigate and then update your failing tests.

Rainforest: ✅ Yes

The Rainforest platform is both intuitive and no-code, so your Test Manager (or anyone else on your team) can create and maintain automated tests up to 3x faster than a QA engineer using open source. So you can keep the release pipeline moving.

When test steps are in plain English, it's easy to quickly interpret and edit them.

Plus, Rainforest’s AI can automatically update — or “heal” — test steps when you make intended changes to your app, which makes the maintenance burden even lighter.

Fast test execution

Hiring: ✅＊ Yes, with caveats

Automated tests are inherently faster than manual tests. But to unlock the true speed potential of automated testing, you’ll want to run your tests in parallel (i.e., simultaneously).

If you want to execute tests in parallel with an open source framework, you either need to provision and configure your own machines, or pay for and configure a 3rd-party test grid like BrowserStack.

The more tests you want to run in parallel, the more complex — or expensive — it gets. As of May, 2024, BrowserStack will charge you $649 per month (paid annually) if you want to run just five desktop browser tests at a time, compared to just $129 for running a single test at a time.  

Rainforest: ✅ Yes

With Rainforest, you can run hundreds of tests in parallel on our cloud-based infrastructure without having to do any configuration. And we don’t charge extra to run tests massively in parallel — it’s included in every plan.

Our customers get automated test suite results in about four minutes, on average.

Efficient communication

Hiring: ✅ Yes

One of the big benefits of hiring is that a QA contributor can integrate tightly with your development team, making communication and task handoffs more fluid and efficient. This keeps releases moving as quickly and smoothly as possible.

Rainforest: ✅ Yes

We’ve set up our solution to emulate the benefits of a full-time hire, making it fast and convenient for your team to work with our contributors.

The Rainforest Test Manager assigned to your account speaks fluent English, works in or near your time zone, embeds in your preferred project and communications tools (e.g., Slack, Teams, Linear, JIRA), and can join your planning meetings. They have the language skills, context, and availability to make communication smooth and efficient. You just tell them what you want tested, and they take care of the rest.

“The flow has been really great. [The Test Managers] are really proactive, they triage all the errors and things that happen in tests, they fix up the tests, they write new tests, and they ask crazy-insightful questions on the product. They’re really polite, cool people. It’s a very similar style to people internally.”

Tyrone Erasmus, Co-founder / CTO at Push Security

Confidence in testing

Reliable service quality

Hiring: ✅＊ Yes, with caveats

Assuming you’ve fortunate enough to have hired well, you can look forward to contributions from a skilled contributor who’s personally invested in the success of your team.

Rainforest: ✅ Yes

All Rainforest Test Managers have been with us since at least 2017 and undergo regular training and evaluations. And they’re all consistently highly rated by customers — otherwise they wouldn’t still be with us.

Software teams working with Rainforest can have the confidence that they’re working with a team with the skills and experience needed to meet high expectations.

“We really didn’t have to put all that much effort into onboarding them. I want to give kudos to our Test Managers for jumping in and more or less figuring the app out on their own. We gave them a long list of things to test. They dove right in and figured it out based on our four-year-old video tutorials.”

Dale Smith, CTO at PayHOA

Understands your product and priorities

Hiring: ✅ Yes

One of the main benefits of hiring is having someone completely dedicated to your team and product who can work more effectively and efficiently exactly because they’re so familiar with your product, team, priorities, and processes.

Rainforest: ✅ Yes

When you opt to include test management services in your Rainforest plan, we’ll dedicate one or more Test Managers to your account. They’ll embed in your workflows and join your project and comms tools (e.g., Slack, Teams, Linear, Jira) and planning meetings, deeply learning your product and priorities.

Just like full-time hires, as they learn, your Rainforest team members will contribute even more effectively and efficiently.

“The fact that our Test Manager is very specialized, very efficient, knows Rainforest incredibly well, and knows our product incredibly well means she can just focus on this one area and do an incredibly good job of it.”

Robert Guillaume, QA Manager at YNAB

Tests the actual user experience

Hiring: ❌ No

QA engineers use open-source frameworks to automate end-to-end tests. These frameworks interact with and evaluate the DOM, or the code behind the visual layer of a web app. That is, they test a proxy for the user experience, but not the user experience itself.

Rainforest: ✅ Yes

Rainforest takes a visual-first approach to interacting with and evaluating your app, so it tests exactly what your users will experience.

Plus, our visual-first approach means you can test anything that appears on a Windows or Mac virtual machine, not just what appears in the browser. For our customers, common use cases outside the browser window include testing browser extensions and file upload/download features.

No black boxes

Hiring: ❌ No

A QA engineer has specific technical expertise to create and maintain automated tests with code. If you don’t have the right skills, you can’t evaluate the quality of their work.

Rainforest: ✅ Yes

Because all Rainforest tests are no-code and readable in plain English, anyone on the team can quickly validate test functionality and coverage.

Test automation for native mobile apps

Hiring: ✅ Yes

A QA engineer can use an open-source framework like Appium to automate tests for native mobile apps on iOS and Android.

Rainforest: ❌ No

Rainforest has been optimized to test web applications — it doesn’t automate tests for native mobile apps. (Though we do offer on-demand crowd testing services for native mobile apps that return scripted test results in under 30 minutes, on average.)

Strategic benefits

Establishes QA process & best practices

Hiring: ? It depends

If you can find an experienced QA person as your first hire, they can do more than just automate your tests. They can implement an overall testing strategy and process, help you define quality metrics, and otherwise help you implement best-practice methodologies.

Rainforest: ✅ Yes

Every Rainforest customer gets a Customer Service Manager (CSM) dedicated to their account. These CSMs aren’t just experts in our services and platform; they’re also QA experts who have helped hundreds of startups develop their QA strategies.

As a consultative partner, your CSM will work with you to design processes and practices for your team that cultivate a culture of quality.

Scalable

Hiring: ❌ No

QA hires often don’t have the bandwidth to do everything they need to — they have to make trade-offs with their time, which means less-robust QA. And it’s not cost-effective to simply add more (expensive) headcount when you want to scale.

Rainforest: ✅ Yes

Rainforest has the resources to give you all the functional QA testing and services you need, so we can scale up with you.

No single-point-of-failure risk

Hiring: ❌ No

A QA hire can represent a single point of failure, particularly if they’re your only QA contributor. If they leave, don’t work out, or are otherwise unavailable, it can bottleneck the entire QA and testing process until you replace them.

Rainforest: ✅ Yes

Rainforest has the personnel and systems in place to guarantee you’ll never be left in a lurch.

Other considerations

• Management overhead: Working with Rainforest Test Managers means not having additional employees to manage. You simply give them the resources to understand your features — many of our customers use informal video walkthroughs — and tell them what you want tested.
• Browser support: The Rainforest platform supports multiple versions of all major browsers.
• Integrations: In addition to integrations with JIRA, Slack, and Microsoft teams, Rainforest has an API and a CLI so you can integrate your automated tests into any CI/CD pipeline.
• Security: Rainforest is SOC 2 Type II certified, HIPAA compliant, and ISO 27002 compliant. We’ve implemented a number of measures and systems to make sure your data is safe both on our platform and with our personnel. This includes having each of our Test Managers sign a non-disclosure agreement (NDA) to protect any information they learn about your products or company. You can learn more on our Security page.
• Exploratory testing: To give you even more confidence in the quality of your app, Rainforest offers exploratory testing services to find the bugs and issues off the “happy paths” covered by your automated tests.