Shop Alto Essentials to add pharmacy products to any delivery!

Your partner in navigating the ins and outs of prescriptions — at no extra cost.

Alto Pharmacy Technician

Alto Pharmacist

It’s official: Alto Pharmacy is a great place to work! We’re honored to join the community of Great Place to Work-Certified™ companies based on team member feedback and a rigorous assessment of our company’s culture and benefits.

Join us in delivering our industry-leading pharmacy care to millions of people across the country. As a member of the team, you’ll play an integral role in improving our customers’ quality of life and redefining what’s possible in pharmacy. 

Be Part of a Great Place to Work — Join Our Team

Hear first-hand from members of the Alto technology teams.

Read the latest Alto blog posts covering healthcare, family health, managing diseases, and other Alto highlights.

Blog | Alto Pharmacy

The Alto product and engineering team has long been focused on initiatives that improve our unit profitability. Now that we’ve laid a strong foundation, we've started to think about our next big bets: How do we expand our services beyond delivering prescriptions? How do we test new value-add offerings quickly and iteratively? That’s where the incubation team comes in.

Meet the Innovation Team Building Alto's Next Big Bets

The Alto product and engineering team has long been focused on initiatives that improve our unit profitability. Now that we’ve laid a strong foundation, we've started to think about our next big bets: How do we expand our services beyond delivering prescriptions? How do we test new value-add offerings quickly and iteratively? That’s where the innovation team comes in.

Newly launched in 2022, the team is small and scrappy by design. We’re a product manager, two engineers, a part-time designer, and a part-time manager with a single mission: to launch at least two significant new business experiments per year. If an experiment is successful, we’ll scale it into a full workstream before handing it off to a stand-alone product team.

We started with over-the-counter (OTC) medications. OTCs, or medications you can buy without a prescription, like Advil, are high-margin products that users are accustomed to purchasing at traditional, brick-and-mortar pharmacies. The goal of this experiment was to assess how much interest users have in adding OTCs to their order while checking out in the Alto app. We set a target conversion rate and offered OTCs only from our San Jose pharmacy, which is less busy than our San Francisco pharmacy but close enough to engineers in San Francisco for in-person troubleshooting.

Our initial hypothesis: if we present users with a small selection of OTCs targeted to their needs (based on the prescriptions in their cart), they’ll be more likely to convert. With support from our pharmacy partners, we carefully curated a recommendation algorithm that would suggest nine OTCs based on the medications already in the user’s cart.

In the interest of lean experimentation, we wanted a scrappy technical solution that could be implemented quickly while meeting our high user experience standards and minimizing operational risk. We settled on the following architecture:

We prioritized creating a clean API for Alto’s customer-facing app and using as much of our current infrastructure as possible. Although OTCs are distinct from prescriptions, we decided to reuse our existing prescription model rather than build a new one from scratch. This allowed us to leverage existing systems and operational workflows, reducing engineering time and risk of regression.

Behind the API, we opted to store our OTC formulary and associated metadata in in-memory data structures:

ADVIL = SkuId.new(sku: '...', sku_type: '...').freeze
CLARITIN = SkuId.new(sku: '...', sku_type: '...').freeze
FISH_OIL = SkuId.new(sku: '...', sku_type: '...').freeze
...
METADATA_BY_SKU_ID = {
  ADVIL.hash_code => {
    image_url: '...',
    name: 'Advil Ibuprofen 200mg Coated Tablets',
    quantity_description: '100 tablets',
    product_description: 'Easy to swallow powerful pain reliever for headaches, joint and muscle pain.',
  },
  CLARITIN.hash_code => {
    image_url: '...',
    name: 'Claritin 24 Hour Allergy Tablets',
    quantity_description: '30 tablets',
    product_description: 'Non-drowsy, 24 hour relief of symptoms due to allergies: sneezing, runny nose, itchy and watery eyes.',
  },
  FISH_OIL.hash_code => {
    image_url: '...',
    name: 'Fish Oil 500mg Softgels',
    quantity_description: '130 softgels',
    product_description: 'Dietary supplement with EPA and DHA omega-3 fatty acids to support heart health and maintain normal triglyceride levels.',
  },
  ...
}.freeze

This gave us a nice balance of speed and flexibility. We knew that we could optimize once we had established our OTC offering and gained an understanding of how it would evolve within Alto.

The results of phase 1 were a bit disappointing, but very interesting: the conversion rate of users who checked out with an OTC was just under our target. However, nearly half of all users who saw the experience would scroll down to peruse all nine OTCs that were offered.

Based on this positive sign of engagement, we hypothesized that perhaps we just weren’t showing the right OTC products to the user. For phase 2, we decided to scrap the limited recommendations and allow users to explore an entire catalog of OTCs.

Phase 2 required categorizing our list of OTCs into buckets such as “Top Sellers,” “Covid Care,” and “Allergy & Asthma.” We built a new fetch_categorized endpoint to return our OTC catalog structured by category, adding search functionality into the client so that users could quickly filter for a specific product. This search field was also heavily tracked with analytics events to help us learn which products to add to our catalog in the future.

This new experience immediately outperformed phase 1. After a few performance improvements (speeding up our catalog and product image load times), we eventually hit our target conversion rate — even managing to surpass it on some weeks!

Our eventing did show that a significant number of users with multiple prescriptions or smaller screens were not even seeing the OTCs in Cart, as they were pushed below the fold. We tried experimenting with alternate spots to show our catalog:

1. An interstitial that shows after Cart and before Checkout - this didn’t perform any better than the control and was quickly scrapped.

2. A single carousel at the top of Cart - this out-performed the control but at the cost of our guardrail metric. With a decrease in the overall checkout rate, we decided to stick with the control for now.

This phase of UI experimentation validated that our users wanted an offering of categorized OTCs. However, our initial technical solution wouldn’t scale as we rolled out to more of our pharmacies. The following questions led us to this conclusion.

How might we support changes to our OTC catalog on a per-pharmacy basis?

Given factors such as inventory, geography-related price sensitivities, and experimentation, we wanted to make sure that we could tweak the OTC experience for users getting deliveries from a specific pharmacy.

In the status quo implementation, that would have likely meant adding an additional layer to our data structures like so:

SKUS_BY_PHARMACY_AND_CATEGORY = {
  SF_PHARMACY_ID => [
    AddOnOtcCategoryType::TopSellers => [
      IHEALTH,
      FLONASE,
      ASPIRIN,
      ...
    ],
    ...
  ],
  ...
}

We weren’t worried about memory space limits given our relatively small number of pharmacies and OTC catalog. We were concerned, however, that each modification would require an engineer to go in and make code changes, get PR approval, and wait for a full deploy for the update to go live. This would be a bad use of engineering time, and our CI/CD process would become a bottleneck to otherwise quick modifications.

How might we add and remove OTCs from our catalog?

Similarly, we knew that adding and removing OTCs from our catalog would also be a somewhat regular workflow. There are some other considerations when adding a new OTC (Is it in our product database? Do we need any new procurement/inventory workflows to make sure it gets stocked in our pharmacies? Do we have a nice image to render in the app?), but we wanted to automate the technical side as much as possible — or at least be able to automate it.

The natural next step was to move our OTC catalog into the database.

Alto’s OTC Database Schema. Some columns excluded for brevity.

Now, rather than grabbing from in-memory data structures, we are efficiently querying the database for the OTC catalog of a user’s specific pharmacy, preloading all of the associated models needed for our API response.

OtcCategory
  .includes(otc_category_products: { otc_product: :otc_product_metadata })
  .where(otc_category_products: { pharmacy_id: pharmacy_id })
  .where(otc_product_metadata: { pharmacy_id: pharmacy_id })


At the time of writing, an engineer is still required to go in and make the changes at the database level. However, we plan to build an admin dashboard so that designated pharmacy partners can tweak our OTC catalog without the help of an engineer.

We’re excited about our next iteration, which will allow users to add OTCs to their upcoming deliveries at any point, not only when checking out. It will likely be live by the time you read this.

Once that’s released, we will continue to learn, iterate, and pay down technical debt — more specifically, maturing the OTC data model in our backend and eliminating the need for them masquerade as prescriptions.

Looking even further ahead, the innovation team has more exciting and impactful projects on our roadmap. We can’t wait to share what’s in store!

While fertility treatments like in vitro fertilization (IVF) have become an increasingly common path to parenthood, helping millions of individuals conceive throughout the past four decades, they remain a significant financial investment.

How Alto Lowers the Cost of Fertility Treatment Through Better Medication Management

Fertility medications used to support egg production, retrievals, and implantations frequently factor heavily into the overall cost of treatment. An average IVF cycle costs between $10,000 and $15,000, with medications an additional $5,000 or more depending on factors like a patient’s age and health history and a clinic’s treatment approach. Unfortunately, a portion of these medications often go unused, as many fertility treatment plans involve last-minute medication changes unsuited for the traditional retail pharmacy model.

As part of our commitment to your financial well-being, we offer a medication management experience that reduces fertility medication waste to lower the cost of treatment, developed in collaboration with Progyny. Take a closer look at our approach.

IVF is a process in which eggs and sperm are fertilized and then placed into the uterus with the potential outcome of a pregnancy. Multiple medications are used on a precisely timed schedule to grow follicles — fluid-filled sacs in the ovaries that potentially contain an egg — retrieve eggs, and, finally, implant a fertilized egg into the uterus. 

Here’s a snapshot of a typical IVF medication schedule:

Days 1-4: stimulation medications such as Gonal-f®, Follistim®, Menopur®, or Low-Dose hCG to stimulate egg production, so that you produce multiple follicles

Days 5-10: continuation of stimulation medications and introduction of antagonists such as Fyremadel™ (ganirelix acetate) and Cetrotide® to prevent premature ovulation

Days 10+: trigger shots of leuprolide acetate, Ovidrel®, and/or Novarel®/Pregnyl® to stimulate final growth spurt and ovulation

After transfer: estrogen and progesterone to support successful implantation of the embryo

Fertility medications are complex, with nightly dosages dependent upon hormone levels measured through bloodwork and ultrasounds. Not only do medication protocols vary from one individual to another, they are typically adjusted throughout the course of a cycle as hormones fluctuate.

Unused medication during fertility treatment

To prevent any missed doses, providers typically prescribe all medications that could be needed in bulk at the beginning of fertility treatment. However, dosages often decrease once a cycle is underway based on a provider’s discretion, ultrasounds, and bloodwork.

At traditional pharmacies, patients undergoing fertility treatment might end up paying for unused medication, as all potential medications are filled upfront. This contributes to the $5.4 billion epidemic of medication waste in the U.S. — one that is particularly acute in fertility, leading some individuals to buy and sell excess medication to each other on social media sites.

Our approach to fertility medication management

Alto has partnered with fertility benefits manager Progyny on an algorithm-powered, app-based medication management experience that reduces unneeded fertility medications while protecting the integrity of time-sensitive and personalized treatment plans.

First, we collect fertility cycle information through the Alto app and run the data through the Progyny algorithm to determine exactly what medication volumes are needed for the next phase of treatment. We then prepare a custom order for just the medications that are immediately needed, which are then delivered directly to your doorstep by our courier team. 

You will receive multiple deliveries throughout the course of your cycle, determined by your specific dosing, trigger schedule and other inputs from your clinical team. Carefully timed check-ins and local courier delivery make last-minute adjustments possible. This approach allows you to manage your medications as your IVF cycle progresses, giving you more control over the total amount of medication that you fill.

Additional financial resources for fertility medications

Alto is committed to helping you save as much as possible on your fertility medications. In addition to our personalized medication management approach, we’ll automatically search for savings and connect you with programs that require you to apply directly, including patient assistance programs from fertility medication manufacturers.

If Alto is your fertility pharmacy, your fertility medication protocol likely includes medications manufactured by EMD Serono or Ferring Pharmaceuticals. EMD has several programs that lower the cost of fertility treatment, including the Compassionate Care program, which offers income-based assistance for fertility medications that aren’t covered by insurance, and special opportunities for military veterans and cancer survivors.

Ferring Pharmaceuticals also offers financial assistance programs to offset the cost of fertility medications, including the HeartTomorrow program for individuals who elect to freeze their eggs, and special opportunities for military veterans and cancer patients.

Read Managing the Costs of Fertility Treatment for more on Alto’s financial support for fertility treatment and medications.

Choose a flexible and reliable pharmacy partner

Fertility treatment can be challenging, but we’ll be with you every step of the way. To ensure that your treatment plan is followed correctly, we offer free same-day delivery of your medications and fertility resources like personalized injection guide videos and one-on-one consultations with fertility-trained pharmacists.

Interested in learning more about how Alto can support your fertility treatment? Request a price quote online or reach out through secure in-app messaging or by phone at 1-800-874-5881.

This content is not intended to be a substitute for professional medical advice, diagnosis, or treatment. Always seek the advice of your physician or another qualified health provider with any questions you may have regarding a medical condition.

All third party trademarks (including logos and icons) referenced by Alto Pharmacy remain the property of their respective owners. Unless specifically identified as such, Alto’s use of third party trademarks does not indicate any relationship, sponsorship or endorsement between Alto and the owners of these trademarks.

When Jake Moffatt says he loves building things, he means it — sometimes literally. A construction project manager turned engineer, Jake has spent the past decade building software at mission-driven organizations. Today, as Engineering Manager of Alto Assistant, he leads the team that’s bringing wizard-like support to Alto’s customer-facing app.

Get to know Jake Moffatt, Engineering Manager on the Alto Assistant Team

When Jake Moffatt says he loves building things, he means it — sometimes literally. A construction project manager turned engineer, Jake has spent the past decade building software at mission-driven organizations. Today, as Engineering Manager of Alto Assistant, he leads the team that’s bringing wizard-like support to Alto’s customer-facing app.
Below, Jake chats with fellow engineering manager Mike Riess and recruiter Braden Hanlon about the work ahead for Alto Assistant.

I’d love to learn more about your background to get us started. How did you end up in software engineering?

When I was twelve, I started learning to code in QBASIC because I wanted to make video games. Once I wrote my first program, I was hooked! I loved computers and the creative power they gave to anyone willing to invest the time and energy to learn. Since then, I’ve built games, personal tools and projects, desktop software, and a number of apps and services — I love building things. My career has gone back and forth between construction and software engineering, but it’s been firmly in the software industry for the last ten years or so.

My favorite place to be in a company is working on something new that has a big impact or addresses an important problem. In a previous role, I built an app that helped people find roommates in a city where housing is prohibitively expensive. At that same company, I led the team responsible for creating a new vertical that eventually split off into its own company.

The goal of Alto Assistant is to help customers help themselves before reaching out to support. Our first project is introducing a wizard-like support “assistant” (hence the team name!) that guides the customer through a series of questions to narrow down the area of support. We’ve already identified a few tasks the assistant can help customers complete on their own, including making changes to upcoming deliveries or scheduling new orders.

In cases where we can’t help them just yet, our new tool generates data to help us to learn exactly what customers need help with. It’s a top priority to bake analytics into every new feature we build; even though we have years of experience supporting our customers — and we know intuitively which problems they frequently experience — we don’t have conclusive data to draw upon just yet.

We plan to start with a couple of frequent use cases and measure how often customers reach out to us for support. Then, we’ll chip away at the problem until we’ve seen a substantial improvement in the user experience. From there, we’ll move on to the next biggest problem and repeat the cycle. There’s a reason “Chip away it” is an Alto value: It’s the playbook that we’ve used to automate other critical parts of the pharmacy value stream like intake, billing, and fulfillment.

What opportunities are you most excited about?

Support drives much of the cost of every shipment today, so this is a high-impact area for our business. A year from now, it would be a stellar outcome if the Assistant team could reduce the number of issues that customers need support on by half.

We’re also thinking about the user experience beyond the app. We plan to improve the tooling that our care specialists use to provide support as well. We’ve already discussed some promising ideas like creating the concept of a conversation in support communications rather than an infinite stream of messages and building out enriched communications. We also intend to develop a custom tool that can expedite a customer’s prescription when they need their medication ASAP.

The possibilities are many, and it will take a team of determined, passionate, empathetic people to solve these challenging problems. I’m super excited to see what the Assistant team will do together.

Any lessons learned from previous experience that you’re applying to your role at Alto?

There’s always the temptation to tackle everything at once with new business initiatives. My experience leading new verticals at Apartment List and Snapdocs taught me the importance of being laser-focused on one or two key metrics instead of trying to build out a fully-realized product from the beginning. That’s exactly what we’re doing on the Alto Assistant Team with our clear focus on reducing communications per shipment — how often customers reach out for help because they could not find what they needed in our app.

It’s also important to recognize that we won’t have all the answers right out of the gate! I like to think of product development as a three-step sequence: learning, growth, and maintenance. Alto Assistant is so young that we’re very much still in the learning phase. We maintain a tight iteration loop with everything we ship: launch, measure the impact, and use the results to iterate and refine our hypotheses about what our users want. Our final product could be anything from a customer support wizard to an AI chatbot to a ticketing system — we have to rely on our key metric of communications per shipment, experimentation, and user feedback to shape the direction.

What got you excited to join Alto? What would you pass on to an engineer thinking about applying to the company?

I was impressed by the Altoids I met during the interview process. I knew I would be working with people from whom I could learn a lot, like Elnaz, who started as an individual contributor and went on to become Director of Engineering and, now, Alto’s first Head of Strategy & Operations. I was also impressed by the quality of the interview process itself — a good sign that the company was well-organized internally. Finally, the pitch for the role was extremely clear, with already-established key metrics that would directly contribute to the company’s bottom line and help Alto become more profitable.

For engineers, I think Alto is a great place for both technical challenges and career growth. We’re still small enough that every individual gets exposure to a broad portfolio of high-impact projects. It’s also a great environment for learning about how a business works — the Product & Technology organization does a great job of sharing information and presenting problems in a way that gets to the root business problem we’re trying to solve. There’s a clear line between any project a team is working on and a foundational business metric. Most importantly, we’re a company that makes a truly positive impact on our users’ lives.

Interested in learning more about engineering at Alto? Follow us on LinkedIn.

Redux is great for managing client state: it allows data to be synchronized across the whole app, reduces prop drilling, and makes it easier to understand when state changes occur.

Server state is a different story. Sure, it’s nice to have data available everywhere in our app, which is why we tend to fetch data from the server and dump it into our Redux store. But because our client application doesn't own that server data, it’s hard to know when it has become stale. And what if multiple components trigger calls to fetch the same data? How do you dedupe requests? How do you decide when to refetch?

Why We Switched to React Query for Managing Server State

Redux is great for managing client state: it allows data to be synchronized across the whole app, reduces prop drilling, and makes it easier to understand when state changes occur.

Server state is a different story. Sure, it’s nice to have data available everywhere in our app, which is why we tend to fetch data from the server and dump it into our Redux store. But because our client application doesn't own that server data, it’s hard to know when it has become stale. And what if multiple components trigger calls to fetch the same data? How do you dedupe requests? How do you decide when to refetch?

Below, we’ll dive into how we’re using React Query at Alto to solve these problems and achieve separation of client and server state.

The Old Way: Fetch Server Data and Store It in State

Historically, we would fetch data and set the response in our React component state.

fetchUser(params) {
 this.setState({ isLoading: true });
 
 return getUser(params)
   .then(({ data }) => {
     this.setState({
       isLoading: false,
       user: data.user,
     });
   })
   .catch(() => {
     toaster.toast({ title: 'Something went wrong...' });
     this.setState({ isLoading: false });
   });
}

To make that data accessible from other components, we’d set that data in the global Redux store. This works fine, except that you have to manage your own loading state, the logic for when and when not to make the request, updating the store when mutating data on the server, and knowing when data is stale.

The React Query Way: Fetch and Cache Server Data

With React Query, fetching data and using that data is more declarative.

const { data, error, isLoading } = useQuery(
 // This is our Query Key, which is unique per query so that React
 // Query can manage the cache for us.
 ['user', { userId }],
 
 // This is our Query Function that actually does the data fetching.
 // We can use anything we want: built-in fetch, axios, etc.
 fetchUser({ userId }),
 
 // This query will not execute until the userId exists.
 { enabled: !!userId },
);

Here, we define our query for a user with a Query Key and a Query Function. The Query Key is unique per query, so React Query can manage our cache for us, and if any part of the Query Key changes, the query will automatically fetch new data. In this example, if the user ID changes, the query will fetch new data for that specific user and store that data in the cache.

Naming is hard, but caching is harder. React Query gives us caching of server data out of the box with cache invalidation and request deduping. If we use this same query with the same Query Key in another component, it'll check the cache: if the data is already there, React Query will simply return it, avoiding extra network requests!

Additionally, useQuery runs wherever you use it — no more fetching data when a component mounts. You just use the query and display the data when it’s ready. The cherry on top? The query returns status updates by default, so we don’t need to store isLoading in component state all the time.

Three Optimizations We Get With React Query

When we rebuilt Alto Connect, our web app for healthcare providers, we found impactful benefits in using React Query to help us manage server state instead of using Redux. Alto Connect provides instant access to patient and prescription information and enables messaging with the pharmacy. Because healthcare providers need to quickly switch between patients and clinics, caching data is important to provide a delightfully fast experience. Apart from the cache, what other optimizations do we get by using React Query? Below are a few that helped us.

Once we’ve queried for some data, it gets entered into our cache, so we can utilize that cached data when using the same query in a different component or when fetching new data for a stale query in the background.

React Query is configured with aggressive but sane defaults and considers cached data as stale. Stale queries will always query for new data when the following criteria are met:

The query is optionally configured with a refetch interval

The fetching for new data when the browser window or tab is refocused is especially useful because the healthcare providers using Alto Connect often switch between multiple browser tabs. If there’s an update to a task, status, or message while they’re checking a patient’s medical chart, it will be immediately reflected in the UI when they come back to Alto Connect.

We also set a refetch interval for our queries involving messages. In the old app, we used WebSockets to enable real-time messaging. Although it’s possible to integrate WebSockets with React Query, we opted for a polling solution instead. Because our app isn’t a real-time messaging application, and we don’t need messages to show up immediately, we set a refetch interval on our messages query to poll for new messages every 30 seconds.

const { data, error, isLoading } = useQuery(
 ['user', { userId }, 'messages'],
 fetchMessages({ userId }),
 {
   // This query is disabled from automatically running if we don’t have a userId
   enabled: !!userId,
 
   // Refetch every 30 seconds
   refetchInterval: 30000,
 
   // Continue to refetch while the tab/window is in the background
   refetchIntervalInBackground: true,
 },
);

This is where the enabled option, which disables a query when it evaluates to false, is particularly powerful. We can pause polling for messages if we route to another page and we know that we won’t make any unnecessary requests if there isn’t a valid userId. We don’t have to worry about closing any socket connections or manually clearing an interval.

Most web apps have some sort of navigation menu, and if you already measure or understand your user behavior, you can prefetch data before users even navigate to a different page. Custom hooks makes it easy to expose a function to prefetch data:

export const usePrefetchUsers = () => {
 const queryClient = useQueryClient();
 const prefetchUsers = async () => {
   await queryClient.prefetchQuery(
     ['users'],
     fetchUsers,
   );
 };
 
 return { prefetchUsers };
};

We prefetch queries when users mouseover or hover over a navigation item so that when they navigate to the next page, the data is either actively being fetched or is already in the cache waiting.

We’ve been talking a lot about queries, but what about mutating data? Mutations, in contrast to queries, are imperative. While queries run automatically when we use them, we need to fire a mutation method in order to run mutations. And in order for our user experience to feel snappy, we can optimistically update the cache and then invalidate it after successfully executing the mutation. useMutation gives us some nice side-effect handlers to deal with every stage of the mutation lifecycle. Now when the user sends a message, that new message is immediately reflected in the UI and we keep the data in sync by invalidating the cache and refetching the messages.

With Redux, we needed to build out solutions to each challenge of managing server state. React Query, on the other hand, gives us those tools right out of the box. We no longer need to worry about when to invalidate or fetch new data, making the user experience more predictable — and the developer experience more enjoyable.

To automate a manual, error-prone pharmacy workflow, we turned to event streaming. Here’s what we learned from one of the first Kafka use cases at Alto.

How We Used Kafka to Build an Event-Driven Automation Pipeline and Scale Our Pharmacy

Imagine this: You're at the doctor’s office, and your doctor just wrote you a new prescription. Minutes later, before you even check out with the receptionist, you already have a notification from Alto Pharmacy on your phone that your prescription is ready. All you have to do is pick the date and time that works best for your free courier delivery.

This is the future we’re building at Alto: a future where a prescription can be transcribed, billed, and verified by a pharmacist in five minutes, flat. To get there without dramatically increasing Alto’s head count, we need to automate much of the process, starting with the first step: transcribing, or intaking, a prescription.

As a pharmacy, we receive prescriptions in many different forms. Some are sent electronically, in a structured format, but others are handwritten, phoned in, or even faxed. To represent all of these forms in a single table, we previously required every incoming prescription to be transcribed by an Alto team member, who would answer questions like:

Whom is this prescription for? Is this customer already in our system?

Where did this prescription come from? Who wrote it?

What is the relevant prescription information (medication to dispense, quantity, days supply, directions, etc.)?

This manual process is both time-consuming and prone to human error, which, if uncaught, could impact our ability to process the prescription downstream. Furthermore, at our busiest times, it could take up to two hours for a team member to get through the pending intake queue — unacceptable if we hope to have prescriptions ready to schedule before people even leave their doctor’s office. 

Automating intake required close collaboration between three teams:

Data Science, who built the decision-making algorithms 

Data Engineering, who served up the results of those algorithms

Product Engineering, who used those results to automatically intake prescriptions 

To allow the teams to operate in parallel without toe-stepping — and in the language most suitable for each (the Data Science and Data teams prefer Python, while product engineers work mainly in Ruby) — we separated the code into two environments: 

1. Data Prod Interface (DPI), where the code for generating automation results lives

2. Wunderbar (our proprietary pharmacy operations platform), where the code for requesting and using those automation results lives

The key question: to establish robust communication between these two environments, should we use an API-based approach, or an event-driven one? Each has its pros and cons. 

Legacy approach everyone at Alto is familiar with

Existing enforcement of strong types and data

Difficult to scale with increased data & requests

Requires ongoing authorization / authentication

Infrastructure relevant to other business areas

Automatic retries, reprocessing, and queueing

Our existing infrastructure currently utilizes a hefty API-based structure, so there was a natural tendency to go down this path, but we knew that beginning to migrate services towards an event-based architecture with Kafka at its heart would help us in the long term.

Kafka is a framework implementation developed by LinkedIn, now open-source with the Apache Software Foundation. It aims to provide a unified, high-throughput, and low-latency platform for handling real-time data feeds. 

Alto uses Amazon Web Services as our cloud provider, and luckily, they offer an out-of-the-box managed solution for Kafka, MSK. To configure Kafka with our system, our infrastructure team worked with AWS to set up managed instances to comply with our business requirements. Our Infrastructure team will share more details about this work in an upcoming blog post. 

Kafka offers many benefits. The asynchronous nature of the message system allows for retries, reprocessing, and queueing as well as easy recovery from outages. In addition, the infrastructure work done for setting up the managed cluster would be a one-off workstream. When complete, we’d be able to re-use the same setup and configurations across various business needs to streamline other projects that leverage Kafka. 

Setting up for success: defining a strict contract

One element we had to get right from the start was the data contract. As we send messages between DPI and Wunderbar, we need to ensure messages are produced with the correct schema, so that downstream consumers know what to expect.

Protocol Buffers, or Protos, is an open-source, cross-platform data format used to serialize structured data. Developed by Google, Protos allow you to define how you structure your data once and then use generated code to easily write and read the structured data in a variety of different languages. 

At Alto, we’ve been using Protos for the past few years to speed up development with auto-generated code and ensure data continuity between our Ruby on Rails backend and JavaScript / React / React Native frontend. Protos may seem antithetical to JavaScript and Ruby since they’re both dynamically typed languages, but in practice, they have saved us countless hours of debugging and back-and-forth Slack messages. 

For this project, we turned again to Protos to define the schema for messages produced by DPI and Wunderbar. We defined IntakeAutomationRequest and IntakeAutomationResult messages in Go and generated the types (Sorbet, Flow, etc.), which both DPI and Wunderbar now refer to when producing or consuming messages.

Because Wunderbar is already configured to run on AWS and houses our existing intake logic, it was easy enough to send a prescription to DPI for automation. Where we would normally create a task for an operations team member to manually transcribe the prescription, we instead produce a message to our “request automation” Kafka topic using the ruby-kafka library.

Our consumer, instantiated in an executable that runs on a dedicated worker, consumes messages from a “receive automation” Kafka topic. First, the consumer verifies that a given message corresponds to a prescription we previously sent to DPI. Then, it persists any automation results included in that message. If the automation results are sufficient, we skip intake altogether. Otherwise, we create an intake task to be handled by an operations team member.

Because this project was among our early use cases for Kafka, we wanted to be prepared for the worst-case scenario: we produce a message requesting automation results, and for whatever reason—maybe DPI is down—we don’t hear back. For us, we defined a reasonable time frame as fifteen minutes, in order to avoid building up a large backlog of work that would have to be offloaded back onto our operations team.
To this end, we store metadata about the automation lifecycle in an intake_automation_results table, which includes timestamps representing when we sent a message to DPI and when we received a response back. A cron job runs every minute to check that table for rows where received_at is blank, and sent_at was more than fifteen minutes ago. If this occurs, we flag the prescription to be manually intaken by an operations team member.

The diagram below illustrates our intake automation architecture.

Intake automation is now live at Alto. We’re seeing prescriptions fly through in just a few minutes — during which time our automation creates a user profile if one doesn’t already exist, links the new prescription to that profile, and finds and bills the user’s insurance; a pharmacist reviews the prescription for clinical accuracy; and the user receives a text message to schedule their delivery. This single flow, prior to intake automation, once took up to four hours to complete!

Going from zero to one is quite a feat, and it wouldn’t have been possible without an event-based approach. Now that intake automation has laid the groundwork, we know that Kafka will only continue to provide exponential value to Alto as we grow and migrate more of our services to this event-based architecture. 

Each type of user has a different product or set of products we’ve built for them. From within each product, we collect, transmit, process, and store a tremendous amount of sensitive information. However, our biggest area of security, privacy, and compliance risk is our internal pharmacy management tool called Wunderbar that is used by our operations (ops) users every day to process prescriptions. Securing access to Wunderbar is mission-critical to protect our data and our business, and, of course, we need our super-secure implementation to be super-smooth to avoid slowing down our hard-working ops users.

Developing Robust Authentication Using Rails, Omniauth, and Okta

Alto Pharmacy aims to deliver a better pharmacy experience for everyone who needs medication. To achieve that, we need to enable three different types of users:

Individual customers who need prescriptions

Healthcare providers: the nurses, doctors, physicians assistants, and other medical professionals who prescribe medications

Operations users: the Altoids who intake prescriptions, bill insurance, fill prescriptions, and more

We wanted to be able to change our Wunderbar session lifetimes without hampering either our security efforts or the usability of our product. Moreover, we wanted session lifetimes of other apps to be able to change appropriately with business risk. Before starting this project, we’d implemented a Google OAuth 2.0 authorization flow that allowed ops users to log into Wunderbar via Google and take advantage of transparent session refresh for a limited amount of time.

However, while Google Workspaces was (and is) great for our email, documents, and calendars, it isn’t as configurable or extensible as a proper SSO solution. One painful side effect of this limitation was that, in order to ensure our refresh sessions were limited to twelve hours in Wunderbar, any user of the Alto Gmail drive also had to log back in every twelve hours just to see email — even though there’s not nearly the same level of sensitive data in email.

Plus, our IT team had already picked the tool of the future: Okta. And the goal was to enable SSO, or Single Sign On: if we still used Google for SSO, it wouldn’t really be SSO!

It was clear we needed to move to Okta for authenticating in Wunderbar, too.

Okta is a big company with many different authentication mechanisms. For us, the big question was really: do we want to authenticate via SAML or Open ID Connect (OIDC)? There are a lot of great resources around comparing these two standards and what they’re good at; here’s a good chart comparing and contrasting parts of these two authentication flows:

From this Medium post, which is a great primer on the differences

For us, the most attractive thing about SAML is that it can be used to provision user accounts from your upstream IDP; for example, Okta could be configured to push username, email, group, role, etc., information to our platforms with which we could automatically provision user accounts.

In the end, we decided the OIDC implementation would provide for a very similar user experience to that which our operations users were already accustomed to, with the same capability to perform transparent token refresh.

This transition was pretty easy because we were moving from Google OAuth 2.0 flow to Okta’s OIDC flow. OAuth 2.0 and OIDC are very similar; just know that OAuth 2.0 is technically an authorization framework (once you have a user, what can they access) vs an authentication framework (actually identifying and validating the user). However, there were some odd things about our previous implementation, and I only had one person to blame for those choices: myself. It’s great to get the opportunity to re-implement our authentication system — in a fast moving startup, you rarely get the chance to revisit problems so soon!

The biggest issue at hand with our previous Google flow was the way in which I’d chosen to re-authenticate users whose Wunderbar sessions had expired. In this case, OAuth provides a handy mechanism: a refresh token. Here’s how access tokens and refresh tokens are exchanged from the client and the authentication server in OAuth 2.0 flows:

At the time, we had session lifetimes of 60 minutes when accessing Wunderbar. These cookies also lasted 60 minutes. We faced a problem: if somebody walked away from their laptop for 61 minutes after authenticating, when they came back, how could we make it so they didn’t have to go back through the Google OAuth 2.0-initiated login flow from the beginning?

My solution at the time persisted a secure cookie with the refresh token value client-side with a longer expiration length. Then, if the user didn’t have a valid access token, we’d simply present the refresh token back to the server and get a new access token all over again. This worked well, but exposing secrets to the client always presents a risk of compromise. Refresh tokens are longer-lived, so they're of even higher value to protect!

Okta’s documentation made it very easy to stand up a simple authentication server and set up a local development server. Okta has a great blog post guide with tools we were already using (Rails and Omniauth, specifically) to get up and running. Beyond that, our biggest challenge was ensuring that we had a smooth rollout.

There was an issue in how we’d test and use this flow in our staging environments, however. At the time we were working on this (early 2020), there were two issues:

Our sandbox environment URLs were non-deterministic and could be varied depending upon how an engineer user named their cluster.

Okta apps did not support wildcard redirects in their oauth redirect URLs. This controls where, upon hitting Okta’s servers, they should redirect your payload to handle the meat of authentication.

For a time, our clever infrastructure engineering team managed around these issues with the use of an intermediary keycloak server. However, Okta now supports wildcard redirects that we’re successfully using today:

The biggest functional change to our authentication flow was in our refresh flow. The biggest challenge in landing this change was in how we deployed it to end users in production.

We didn’t want to expose the refresh token because it presented too much risk. What do we do instead? Something a little bit risky, but that we preferred after threat modeling our scenarios. Security engineering is often about making tradeoffs given a number of constraints. Today, our session lifetimes are down to fifteen minutes, and our refresh lifetime for an active session (e.g. somebody making at least one request within this window) is 60 minutes. If somebody comes back after their access token expires, that’s fine; we’ve persisted the encrypted refresh token to the database, and we can use attributes from the expired but still cryptographically valid token in order to look that refresh token up.

This was crucial for Okta especially because Okta refresh tokens are valid for 100 days by default!

Disruptive change is hard. We didn’t want to disrupt the workday of our hard-working ops users who are caring for our many customers and provider partners. The biggest intent here was to keep a safety blanket fallback to the old authentication flow until we were quite confident in its functionality. This broke down into a multi-phase approach:

1. Dark-launch the new Okta flow hidden behind a URL and with a small allow-list of users (mostly just the Security team) assigned to the new application in Okta.

2. Ask for volunteers! These volunteers would go about their normal work day in a variety of functional areas (customer care, pharmacy, fulfillment, engineering) using Okta instead of Google.

3. Change the default login flow to use Okta, but keep Google working just fine as a secondary action hidden behind a URL.

This allowed us to always ensure that we didn’t disrupt peoples’ days too much!

We achieved our goals and now, nearly two years later, we’re successfully using Okta’s OIDC flow within our core pharmacy management app. What did we learn in the process?

Not every API is necessarily fully implemented in every client. At the time of implementation, Okta’s Ruby client wasn’t fully supported. In order to support our refresh flow, we needed to upstream a small change into the client library, which required a little investigation.

Exposing secrets involves risk! In our case there wasn’t a glaring need to expose refresh tokens client-side at all.

Slow rollouts are great, especially with disruptive changes! A big thank-you to our team members who were willing to be part of the experiment group last year when this rolled out.

Interested in learning more about engineering at Alto? Follow us on LinkedIn and check out our open positions.

Last year, we set out to tackle this technical debt by splitting the User model into separate concerns and refactoring the entire system into this new paradigm.

Tackling Technical Debt and Improving Security: How Alto Untangled an Overloaded User Model

At Alto, we have three types of users: customers, healthcare providers (nurses, doctors, etc), and operations users who run the pharmacy. For each type of user, we maintain a separate application (though they share a monolithic backend for now): our customer app; our Alto Connect app for providers; and our internal pharmacy management tool called Wunderbar, which powers the pharmacy experience from prescription intake to fulfillment.

Early on, we made the pragmatic choice to represent both customers and operations users with a single User model, distinguishing them with a “roles” attribute. This required repetitive checks that, over time, slowed down our engineering teams, presented substantial privacy risk, and posed difficult scaling challenges as we evolved our monolith into different microservices. 

The challenges of relying on a single User model for different types of users

Blog

Meet the Innovation Team Building Alto's Next Big Bets

How Alto Lowers the Cost of Fertility Treatment Through Better Medication Management

Get to know Jake Moffatt, Engineering Manager on the Alto Assistant Team

Why We Switched to React Query for Managing Server State

How We Used Kafka to Build an Event-Driven Automation Pipeline and Scale Our Pharmacy

Developing Robust Authentication Using Rails, Omniauth, and Okta

Tackling Technical Debt and Improving Security: How Alto Untangled an Overloaded User Model

Considerations and Approaches for Implementing Biometric Login: Part 1

How to Take Action for World AIDS Day 2021

Introducing Alto Connect

The Next Evolution of Our Company Values

The Role of Insulin Pumps in Diabetes Treatment

Switch to a better pharmacy.