October 26, 2022
Five hundred percent growth. That was the goal Dealerware set for itself.
Dealerware provides a solution that modernizes and streamlines fleet management for automotive retailers. The company planned a set of growth initiatives intended to drive up rental and loaner contract volume and quintuple the number of vehicles under management.
Prior to these initiatives, however, Dealerware’s engineering team saw usage spikes at 8 AM for pick-up and around 5 PM for drop-off. To prepare for the anticipated growth and even greater spikes during peak demand, the engineering team migrated Dealerware’s platform from monolithic applications to a more scalable container-based architecture.
“It’s great to have something to be able to trace the root of the problem at the infrastructure view. It’s provided insights into issues I wasn’t aware of.”
Kenneth Skertchly
Senior DevOps Engineer, Dealerware
Along with this transition, Dealerware sought new tools for streamlining and scaling operations. With exponential growth and a new architecture, the team was concerned about possible bottlenecks, performance issues and latency of Dealerware applications.
App performance and speed became even more critical when the COVID-19 pandemic created an additional, urgent requirement: the need for a contactless experience.
Ensuring top-of-the-line service
For Dealerware, app performance is mission critical. Dealerships depend on the Dealerware platform’s capabilities and responsiveness. To deliver the best possible customer experience, the company wanted to better measure and understand application performance.
Since its founding in 2016, Dealerware has been running on the Amazon Web Services (AWS) cloud platform, building its app on Amazon Elastic Compute Cloud (EC2) instances with a monolithic stack. Containers are smaller resources that are faster to provision than plain Amazon EC2 instances. Provisioning containerized applications can take just seconds — sometimes milliseconds — versus minutes for virtual machines. This speed matters. With containers, Dealerware can scale more effectively and go from five to forty instances quickly when demand spikes, rather than experiencing latency and degrading the customer experience during peak hours.
But monitoring application performance in a container-based architecture is impossible without the right tools. Dealerware needed greater visibility across its containers, API calls and database queries in order to uncover bottlenecks. It needed a way to visualize the relational dependencies of every application and infrastructure component. And the dependencies between services can go many layers deep. A latency spike in a single microservice can cascade up and downstream, causing issues.
End-to-end visibility
When the product team brought the goal of 500% growth to the engineering team, the initial response was, “We’ll need to monitor it. How do we do that best?”
The Dealerware DevOps team remembered seeing a demo of IBM® Observability by Instana technology at a DevOps Days conference a couple years prior. The Instana application monitoring solution offered an seamless launch, with a single agent that automatically detects what’s running and collects data from it.
Once the Instana agent has been installed into Dealerware’s Amazon Elastic Kubernetes Service (EKS) clusters, it automatically discovers all the containers running in the pods, services and endpoints, and the dependencies between them. The agent automatically captures and indexes error messages against each endpoint and rolls them up to the service and application level.
The Instana solution also offers monitoring for the microservices and cloud-native technologies, such as EKS and the full AWS stack that Dealerware now uses. The customizable Instana dashboards provide rich visualization of application dependencies and performance metrics. The flexibility is important for potential future target users beyond the DevOps and product teams.
After moving to a better performing environment, observability, monitoring and scaling abilities were just what Dealerware needed to control latency.
Generally, observability tools collect and display data from the system that teams want to monitor. But data requires meaningful and actionable analysis. The better your analysis capabilities are, the more valuable your investments in observability and monitoring become.
That is where Instana shines. The Instana Enterprise Observability Platform offers automatic detection of whole environments, with comprehensive monitoring of EKS clusters. The Instana agent does all the heavy lifting without additional configuration:
- Auto-injection into containers at runtime
- Rich visualization of application dependencies and performance metrics
- Comprehensive mapping of all application dependencies
There is a trace for every request — no sampling. Instana provides full visibility, monitors end-user transaction requests and provides the relevant correlated back-end requests.
Instana also enables alerting across three categories:
- Changes — small events such as changes to container labels or hosts spinning up or down
- Issues — low-level events such as CPU usage spikes or container limits being reached
- Incidents — more complex sets of events, including machine learning informing when an application looks less-than-perfect, with a historical view to show what led to the incident
These alerts provide insight into what occurred before troubleshooting even begins. It’s not just application performance management (APM), but a real-time infrastructure monitoring, platform monitoring, end-user monitoring and intelligent alerting system.
Dealerware stops latency in its tracks
With Instana, Dealerware can now monitor, observe and manage all of its environments. From a single control pane, users can see where issues occur, understand the causes and initiate fixes.
“I’ve been using Instana as a troubleshooting tool,” says Kenneth Skertchly, a Senior DevOps Engineer at Dealerware. “It’s great to have something to be able to trace the root of the problem at the infrastructure view. It’s provided insights into issues I wasn’t aware of.”
Dealerware now understands why a given database query or microservice spiked in latency, and how to remove the bottleneck. With this information and action, the company closes latency gaps as they occur, improving the Dealerware platform’s performance and the customer experience. When a specific service has latency issues, the engineering team can alert the UI team to hide certain functionality.
This capability was important particularly after the industry change near the beginning of 2020, when Dealerware had to adjust standard operating procedures and go contactless. The company discovered that in some cases, contracts sent via SMS were taking up to 10 minutes to get to customers.
Using real-time data from Instana and AWS CloudWatch, the DevOps team discovered that the text messages were queued with longer running jobs. The team was able to separate the queues, monitor them and set alerts, resulting in a decrease in delivery latency from 10 minutes down to 10 – 12 seconds.
What’s next for Dealerware
With latency under control, the Dealerware team is shifting its focus to building out new platform features and functionality, and accelerating its continuous integration, continuous delivery (CI/CD) process.
Bryce Hendrix, Lead Platform Architect at Dealerware, explains: “With Instana, our day-to-day goal is to be able to guarantee a latency expectation. Our goal for service calls is to complete within less than 250 milliseconds. So, it’s not just for fire drills. In the day-to-day, we’re able to improve performance, and that drives us toward that 250 milliseconds goal. Instana makes this possible.”
The engineering team is also looking to expand use of Instana to other teams, in particular marketing and customer support. While these teams may not want to drill down as deep into data as development and operations, higher-level metrics relevant to their business will yield valuable insights.
This is where Instana’s customizable dashboards come in. Rather than looking at one single monitor in the office, individual users can create widgets that show custom metrics relevant for them, and assemble them to create tailored landing pages that will be the first thing they see in the morning when work starts.
About Dealerware
DealerwareExternal Link provides a connected-car mobility solution that streamlines fleet management at automotive dealerships. Its platform allows dealers to lower cost and enhance the customer experience in the service department. Today, Dealerware manages tens-of-thousands of vehicles for hundreds of dealerships, working with all of the top 10 dealer groups and more than 25 original equipment manufacturer brands.
About Instana, an IBM Company
Instana, an IBM Company, provides an Enterprise Observability Platform with automated application performance monitoring capabilities to businesses operating complex, modern, cloud-native applications no matter where they reside—on premises or in public and private clouds, including mobile devices or IBM Z® mainframe computers.
Control modern hybrid applications with Instana’s AI-powered discovery of deep contextual dependencies inside hybrid applications. Instana also provides visibility into development pipelines to help enable closed-loop DevOps automation.
These capabilities provide actionable feedback needed for clients as they optimize application performance, enable innovation and mitigate risk, helping DevOps increase efficiency and add value to software delivery pipelines while meeting their service and business level objectives.
For more information, visit instana.com.
Solution component