Automation QA Engineer
Transition Technologies MS
This role is for an experienced QA automation engineer who will design and implement test strategies for both traditional full-stack applications and AI/ML systems. Day-to-day includes writing automated tests in Python (pytest), using Playwright/Cypress for UI, API testing, and performance testing. A key differentiator is the focus on testing AI outputs, which requires understanding of LLM evaluation benchmarks and validation of non-deterministic content. The role also involves enabling developers to write their own tests (shift-left) and ensuring quality gates for content generation pipelines.
Brak jawnych widełek — wynagrodzenie do ustalenia podczas rekrutacji.
Brakuje: no information about team size or composition., no details on the recruitment process (number of stages, technical tasks)..
This role is for an experienced QA automation engineer who will design and implement test strategies for both traditional full-stack applications and AI/ML systems. Day-to-day includes writing automated tests in Python (pytest), using Playwright/Cypress for UI, API testing, and performance testing. A key differentiator is the focus on testing AI outputs, which requires understanding of LLM evaluation benchmarks and validation of non-deterministic content. The role also involves enabling developers to write their own tests (shift-left) and ensuring quality gates for content generation pipelines.
- ✓Focus on cutting-edge AI/ML testing, including LLM evaluation and non-deterministic output validation.
- ✓Shift-left quality approach empowers developers and reduces bottleneck on QA.
- ✓Includes performance and load testing responsibilities, offering diverse technical challenges.
- !Occasional meetings with US team may require flexibility in working hours (e.g., early mornings or late evenings).
- !Role requires designing test strategies for AI/ML systems, which may be vague without concrete tools or frameworks specified.
- ?Brak jawnych widełek — wynagrodzenie do ustalenia podczas rekrutacji
- •Writing automated UI tests using Playwright or Cypress
- •Creating and maintaining API test suites for FastAPI endpoints
- •Designing test strategies for LLM outputs including golden-set validation
- •Integrating automated tests into CI/CD pipelines (Azure/AWS)
- •Performing load and performance testing on platform services
- •Collaborating with AI engineers to define quality gates for content generation
- •Reviewing unit and integration tests written by developers
- •Debugging and troubleshooting test failures in microservice architecture
Oferta dla doświadczonych specjalistów (Senior).
A QA engineer with at least 3 years of experience, solid Python and pytest skills, some exposure to Playwright or Cypress, and basic API testing. They may have less AI experience but are eager to learn and adapt to testing non-deterministic systems.
Junior candidates with less than 3 years of experience, or manual testers without automation skills. Also not for those who dislike defining test strategies or working with AI/ML systems.
- ?How large is the QA team and how is it structured?
- ?What specific AI/ML projects or platforms will I be working on?
- ?What tools or frameworks are used for LLM evaluation and golden-set validation?
- ?How is the shift-left model currently implemented, and what is the expected level of developer ownership?
- ?Are there on-call duties or expectations for overtime?
- ?What is the budget for training or conferences related to AI/ML testing?
- ?How often are meetings with the US team, and what is the typical time overlap?
- −No information about team size or composition.
- −No details on the recruitment process (number of stages, technical tasks).
- −No specific benefits package details beyond 'attractive'.
- −No mention of on-call or after-hours support requirements.
Non-corporate, flexible environment with a focus on innovation in AI/ML testing. The team likely values autonomy and collaboration across QA, development, and AI engineering.