Managed Test Services for a leading US-based low cost carrier

Client

A leading American airline based in Dallas, Texas, is the world's largest low-cost carrier with a fleet of 752 aircraft. It employs approximately 60,000 individuals and facilitates around 4,000 daily departures during peak travel periods, generating revenue nearing $21.965 billion.

Business Context

The airline was looking at consolidating vendors within the central Quality Management Organization. This aims to establish a singular point of accountability, ensuring consistency and standardization across tools, processes, and governance. It seeks long-term cost optimization and fosters innovation in quality engineering. Anticipated business advantages include predictable outcomes in both cost and quality, as well as expedited service delivery to end customers.

Engagement Highlights

Peak team size: 300+

Duration of engagement: 11+ Years

Client location: US (Dallas)

On: Off ratio: 45:55

Onshore location: Dallas

Offshore location: Bangalore

Number of Apps: 58+

Tech Stack/Applications: Web Services Automation(SOAP/REST), Cloud Testing, Web Application Automation, Stand Alone Application Automation, CI/CD, Version Control, Device Testing & Automation Mobile (iOS, Android, mWeb),iPAD, E2E Automation, JAVA, Python, Groovy, VBScript, Shell, NFR Testing, ETL Testing.

Tools: ALM, Jira, Git, BitBucket, SVN, Intellij, PyCharm, Gradle, Maven, Jenkins, LeanFT, UFT, SeeTest, Selenium-Java, AutoIT, SoapUI, ApacheCxf , RestAssured, PostMaster, TestNG, Serenity, JBehave, Jeb, Putty, Performance Centre (Load Runner), Grafana, Teradata, Ab Initio

Business Challenges

Minimal business disruption during transition:

• The client was concerned about the transition's complexity, especially regarding in-flight releases 

High cost 

• Checking 58 applications with various technologies for their flagship GDS migration program 

Quality & efficiency 

• Inconsistent testing standards across commercial application and aircraft maintenance portfolios 
• Supporting multiple release testing and interdependencies requires managing data and environment dependencies 
• Lack of a scalable enterprise test strategy for performance engineering and automation 
• Unavailable centralized test governance and SLAs to monitor delivery 

LTM Solution

Minimal business disruption during transition

• LTM established a Quality Assurance (QA) Centre of Excellence (COE). Within a period of 6 weeks, they ramped up over 130 people, both domestically and offshore. They completed the transition for 58 applications within 8 weeks with 0% overrun 
• Focus on critical applications as Wave 1 for transition aims to minimize vendor dependency and mitigate early vendor attrition 
• Front-loading the team for in-flight projects during the transition led to a shorter transition period 
• Training for testers without prior experience in the airline domain enabled them to ramp up within 3-6 weeks, depending on the complexity of the applications 
• Leadership commitment from both client and LTM to mitigate risks like vendor attrition 

High cost : 

• LTM delivered a significant cost reduction in the form of increased automation coverage from 40% to 70% 
• Implemented process and resourcing efficiencies through a core-flex model, resulting in reductions in team size 
• Over 8 months, the shift to a Managed Test Function saved about $1.05 million over eight months. This included end-to-end testing ownership for critical applications such as passenger service systems, revenue management, digital, and flight network planning, covering both functional testing and test automation 
• $125,000 savings in tool cost by moving to open source in the mobile space 
• 60% saving on PNR data creation and validation efforts 

Quality & efficiency 

• Shrunk release cycle time from 12 to 8 weeks by implementing enterprise test automation across multiple tracks such as .Com, reservation system and mobile application leveraging LTM accelerators and building CI/CD pipeline through continuous testing 
• Automation and performance testing assets baselining helped to define the automation & performance roadmap 
• Set up dedicated ED&A (Enterprise Data & Architecture) team to validate the correctness of data loading and transformation through systematic ETL testing 
• Established an integration/E2E testing team to instill the confidence needed for airlines to promote key changes to production 
• Set up governance, communication and reporting practices that include service level, financial & quality dashboards 
• Set up robust QA metrics-driven governance model with pre-defined escalation channels to track a delivery and committed SLAs 

Innovation in quality engineering

• Service virtualization to build 3rd party services and reduce the cost of testing and increase time to market 
• Defect prediction using AI-ML to reduce the number of expected defects in future releases 
• Shift left performance engineering thereby identifying performance bottlenecks early 
• Operational resilience testing ensuring 99.99% availability of critical systems 
• Impact based analysis of code changes reducing overall regression test execution time 
• Chat bot for automated test execution as a self-service tool to DevOps and CloudOps team to execute test cases based on natural language commands 
• Automation script failure analysis and screen capture utilities were built to reduce cycle time

Business Benefits

Transition with minimal disruptions and overrun 

• Easy transition and ramp up for 58 such business critical applications which involved taking over the work of 150+ testers and automation engineers 
• Created a virtual learning platform that spans over 100+ topics, including dotcom, customer management, revenue, and non-revenue bookings portfolios. This platform facilitates a faster and seamless ramp-up of new testers within just three weeks

High cost 

• Deployed an end-to-end automation strategy across multiple tracks resulting in increase of automation coverage from 40% to over 70% thereby reducing the manual testing efforts and cost 
• End-to-End automation suite aided in identifying over 500 environment blockers within a year 
• 12% of functional defects detected in development phase due to shift left approach 
• This helped in the development teams react to the defects early in the lifecycle 
• Achieved cost savings of $1.56M within the first 12 months through right shoring and through moving towards One Engineering team – Shift Left Testing & conversion of functional testers to SDETs 
• Through efficiency, right shoring and cost reduction saved an initial $7M of the annual budget 

Quality & efficiency 

• Achieved a high DRE of over 98% till date 
• Created a quality dashboard on Power BI platform that would enable real time reporting 
• Consistently met the SLAs on productivity gains, quality, automation coverage, test coverage, and time to market 
• Reduction of test execution cycle time by 66% in the commercial application area 
• Saved ~$250K by virtualizing critical 3rd party services 
• Test management office set up to drive delivery and innovation governance with the head of the client's Quality Management Organization 
• Implemented service virtualization, enterprise automation, performance engineering (aircraft operations), and test data management to transform test engineering, aiming to improve test efficiencies