Your partner in navigating the ins and outs of prescriptions — at no extra cost.

Alto Pharmacy Technician

Alto Pharmacist

It’s official: Alto Pharmacy is a great place to work! We’re honored to join the community ofGreat Place to Work-Certified™ companies based on team member feedback and a rigorous assessment of our company’s culture and benefits.

Join us in delivering our industry-leading pharmacy care to millions of people across the country. As a member of the team, you’ll play an integral role in improving our customers’ quality of life and redefining what’s possible in pharmacy.

Be Part of a Great Place to Work — Join Our Team

Hear first-hand from members of the Alto technology teams.

Read the latest Alto blog posts covering healthcare, family health, managing diseases, and other Alto highlights.

Blog | Alto Pharmacy

With one in ten Americans living with type 1 or type 2 diabetes and more than 88 million adults living with prediabetes, many more people have a connection to diabetes than it may seem. And whether you are living with diabetes yourself or care about someone who is, or if you simply want to know more about diabetes prevention, there’s no shortage of information to learn or actions to take. Thankfully, there are a variety of resources that can help, from online support groups and events to diabetes education programs and advocacy materials. Explore our diabetes resource round-up below to get started.

Diabetes Resource Round-up

With a mission of improving the lives of everyone affected by diabetes, the American Diabetes Association (ADA) is a comprehensive hub for diabetes research, education, advocacy, and support. The national organization also has local offices throughout the U.S. where you can learn about community resources and events.

Beyond Type 1 and Beyond Type 2 are global online communities for individuals living with diabetes, powered by a nonprofit organization (also named Beyond Type 1) that funds a variety of diabetes research and projects.

Focusing on type 1 diabetes, JDRF is a leader in research and advocacy and a destination for type 1 diabetes support and community. Like the ADA, JDRF has many chapters across the U.S. that offer opportunities to connect locally with those affected by type 1 diabetes.

Founded by an endocrinologist living with type 1 diabetes himself, Taking Control of Your Diabetes is a resource hub for individuals and families affected by diabetes as well as medical professionals seeking to stay informed of the latest developments in diabetes care.

ADA engagement platform, with opportunities to take action and raise awareness

Association of Diabetes Care & Education Specialists (ADCES) legislative action center

Beyond Type 1 community table, a monthly conversation series

ADCES diabetes education program directory

JDRF type 1 diabetes daily management page

Getinsulin.org, an insulin access resource

Johnson & Johnson Patient Assistance Program

Lilly Cares Foundation Patient Assistance Program

Novo Nordisk Diabetes Patient Assistance Program

ADA tips on nutrition, fitness, mental health, and more

ADA free type 1 diabetes self-care manual

JDRF type 1 diabetes health insurance guide 

Project Power, the ADA’s free virtual after-school diabetes prevention program for children

ADA summer camps and year-round programming for children with diabetes

ADA resources on school safety and diabetes

With a dedicated diabetes support team, Alto has deep expertise in diabetes management. Our extensive library of diabetes-focused content offers the information you need to manage the condition and live an active life. Here are a few of our top reads:

What’s the Difference between Type 1 and Type 2 Diabetes?

New Diabetes Diagnosis? Here’s What to Know

How to Manage Your Child’s Type 1 Diabetes Diagnosis

We make it simple to manage your diabetes, with free, same-day delivery right to your doorstep, temperature controlled packaging, and automatic savings investigations. And our dedicated diabetes team is here to offer individualized support and walk you through your first insulin doses.

Reach out via text or phone at 1-800-874-5881 or in-app messaging to learn how Alto can be a partner in your diabetes care.

This content is not intended to be a substitute for professional medical advice, diagnosis, or treatment. Always seek the advice of your physician or another qualified health provider with any questions you may have regarding a medical condition.

To automate a manual, error-prone pharmacy workflow, we turned to event streaming. Here’s what we learned from one of the first Kafka use cases at Alto.

How We Used Kafka to Build an Event-Driven Automation Pipeline and Scale Our Pharmacy

Imagine this: You're at the doctor’s office, and your doctor just wrote you a new prescription. Minutes later, before you even check out with the receptionist, you already have a notification from Alto Pharmacy on your phone that your prescription is ready. All you have to do is pick the date and time that works best for your free courier delivery.

This is the future we’re building at Alto: a future where a prescription can be transcribed, billed, and verified by a pharmacist in five minutes, flat. To get there without dramatically increasing Alto’s head count, we need to automate much of the process, starting with the first step: transcribing, or intaking, a prescription.

As a pharmacy, we receive prescriptions in many different forms. Some are sent electronically, in a structured format, but others are handwritten, phoned in, or even faxed. To represent all of these forms in a single table, we previously required every incoming prescription to be transcribed by an Alto team member, who would answer questions like:

Whom is this prescription for? Is this customer already in our system?

Where did this prescription come from? Who wrote it?

What is the relevant prescription information (medication to dispense, quantity, days supply, directions, etc.)?

This manual process is both time-consuming and prone to human error, which, if uncaught, could impact our ability to process the prescription downstream. Furthermore, at our busiest times, it could take up to two hours for a team member to get through the pending intake queue — unacceptable if we hope to have prescriptions ready to schedule before people even leave their doctor’s office. 

Automating intake required close collaboration between three teams:

Data Science, who built the decision-making algorithms 

Data Engineering, who served up the results of those algorithms

Product Engineering, who used those results to automatically intake prescriptions 

To allow the teams to operate in parallel without toe-stepping — and in the language most suitable for each (the Data Science and Data teams prefer Python, while product engineers work mainly in Ruby) — we separated the code into two environments: 

1. Data Prod Interface (DPI), where the code for generating automation results lives

2. Wunderbar (our proprietary pharmacy operations platform), where the code for requesting and using those automation results lives

The key question: to establish robust communication between these two environments, should we use an API-based approach, or an event-driven one? Each has its pros and cons. 

Legacy approach everyone at Alto is familiar with

Existing enforcement of strong types and data

Difficult to scale with increased data & requests

Requires ongoing authorization / authentication

Infrastructure relevant to other business areas

Automatic retries, reprocessing, and queueing

Our existing infrastructure currently utilizes a hefty API-based structure, so there was a natural tendency to go down this path, but we knew that beginning to migrate services towards an event-based architecture with Kafka at its heart would help us in the long term.

Kafka is a framework implementation developed by LinkedIn, now open-source with the Apache Software Foundation. It aims to provide a unified, high-throughput, and low-latency platform for handling real-time data feeds. 

Alto uses Amazon Web Services as our cloud provider, and luckily, they offer an out-of-the-box managed solution for Kafka, MSK. To configure Kafka with our system, our infrastructure team worked with AWS to set up managed instances to comply with our business requirements. Our Infrastructure team will share more details about this work in an upcoming blog post. 

Kafka offers many benefits. The asynchronous nature of the message system allows for retries, reprocessing, and queueing as well as easy recovery from outages. In addition, the infrastructure work done for setting up the managed cluster would be a one-off workstream. When complete, we’d be able to re-use the same setup and configurations across various business needs to streamline other projects that leverage Kafka. 

Setting up for success: defining a strict contract

One element we had to get right from the start was the data contract. As we send messages between DPI and Wunderbar, we need to ensure messages are produced with the correct schema, so that downstream consumers know what to expect.

Protocol Buffers, or Protos, is an open-source, cross-platform data format used to serialize structured data. Developed by Google, Protos allow you to define how you structure your data once and then use generated code to easily write and read the structured data in a variety of different languages. 

At Alto, we’ve been using Protos for the past few years to speed up development with auto-generated code and ensure data continuity between our Ruby on Rails backend and JavaScript / React / React Native frontend. Protos may seem antithetical to JavaScript and Ruby since they’re both dynamically typed languages, but in practice, they have saved us countless hours of debugging and back-and-forth Slack messages. 

For this project, we turned again to Protos to define the schema for messages produced by DPI and Wunderbar. We defined IntakeAutomationRequest and IntakeAutomationResult messages in Go and generated the types (Sorbet, Flow, etc.), which both DPI and Wunderbar now refer to when producing or consuming messages.

Because Wunderbar is already configured to run on AWS and houses our existing intake logic, it was easy enough to send a prescription to DPI for automation. Where we would normally create a task for an operations team member to manually transcribe the prescription, we instead produce a message to our “request automation” Kafka topic using the ruby-kafka library.

Our consumer, instantiated in an executable that runs on a dedicated worker, consumes messages from a “receive automation” Kafka topic. First, the consumer verifies that a given message corresponds to a prescription we previously sent to DPI. Then, it persists any automation results included in that message. If the automation results are sufficient, we skip intake altogether. Otherwise, we create an intake task to be handled by an operations team member.

Because this project was among our early use cases for Kafka, we wanted to be prepared for the worst-case scenario: we produce a message requesting automation results, and for whatever reason—maybe DPI is down—we don’t hear back. For us, we defined a reasonable time frame as fifteen minutes, in order to avoid building up a large backlog of work that would have to be offloaded back onto our operations team.
To this end, we store metadata about the automation lifecycle in an intake_automation_results table, which includes timestamps representing when we sent a message to DPI and when we received a response back. A cron job runs every minute to check that table for rows where received_at is blank, and sent_at was more than fifteen minutes ago. If this occurs, we flag the prescription to be manually intaken by an operations team member.

The diagram below illustrates our intake automation architecture.

Intake automation is now live at Alto. We’re seeing prescriptions fly through in just a few minutes — during which time our automation creates a user profile if one doesn’t already exist, links the new prescription to that profile, and finds and bills the user’s insurance; a pharmacist reviews the prescription for clinical accuracy; and the user receives a text message to schedule their delivery. This single flow, prior to intake automation, once took up to four hours to complete!

Going from zero to one is quite a feat, and it wouldn’t have been possible without an event-based approach. Now that intake automation has laid the groundwork, we know that Kafka will only continue to provide exponential value to Alto as we grow and migrate more of our services to this event-based architecture. 

Interested in learning more about engineering at Alto? Follow us on LinkedIn.

Each type of user has a different product or set of products we’ve built for them. From within each product, we collect, transmit, process, and store a tremendous amount of sensitive information. However, our biggest area of security, privacy, and compliance risk is our internal pharmacy management tool called Wunderbar that is used by our operations (ops) users every day to process prescriptions. Securing access to Wunderbar is mission-critical to protect our data and our business, and, of course, we need our super-secure implementation to be super-smooth to avoid slowing down our hard-working ops users.

Developing Robust Authentication Using Rails, Omniauth, and Okta

Alto Pharmacy aims to deliver a better pharmacy experience for everyone who needs medication. To achieve that, we need to enable three different types of users:

Individual customers who need prescriptions

Healthcare providers: the nurses, doctors, physicians assistants, and other medical professionals who prescribe medications

Operations users: the Altoids who intake prescriptions, bill insurance, fill prescriptions, and more

We wanted to be able to change our Wunderbar session lifetimes without hampering either our security efforts or the usability of our product. Moreover, we wanted session lifetimes of other apps to be able to change appropriately with business risk. Before starting this project, we’d implemented a Google OAuth 2.0 authorization flow that allowed ops users to log into Wunderbar via Google and take advantage of transparent session refresh for a limited amount of time.

However, while Google Workspaces was (and is) great for our email, documents, and calendars, it isn’t as configurable or extensible as a proper SSO solution. One painful side effect of this limitation was that, in order to ensure our refresh sessions were limited to twelve hours in Wunderbar, any user of the Alto Gmail drive also had to log back in every twelve hours just to see email — even though there’s not nearly the same level of sensitive data in email.

Plus, our IT team had already picked the tool of the future: Okta. And the goal was to enable SSO, or Single Sign On: if we still used Google for SSO, it wouldn’t really be SSO!

It was clear we needed to move to Okta for authenticating in Wunderbar, too.

Okta is a big company with many different authentication mechanisms. For us, the big question was really: do we want to authenticate via SAML or Open ID Connect (OIDC)? There are a lot of great resources around comparing these two standards and what they’re good at; here’s a good chart comparing and contrasting parts of these two authentication flows:

From this Medium post, which is a great primer on the differences

For us, the most attractive thing about SAML is that it can be used to provision user accounts from your upstream IDP; for example, Okta could be configured to push username, email, group, role, etc., information to our platforms with which we could automatically provision user accounts.

In the end, we decided the OIDC implementation would provide for a very similar user experience to that which our operations users were already accustomed to, with the same capability to perform transparent token refresh.

This transition was pretty easy because we were moving from Google OAuth 2.0 flow to Okta’s OIDC flow. OAuth 2.0 and OIDC are very similar; just know that OAuth 2.0 is technically an authorization framework (once you have a user, what can they access) vs an authentication framework (actually identifying and validating the user). However, there were some odd things about our previous implementation, and I only had one person to blame for those choices: myself. It’s great to get the opportunity to re-implement our authentication system — in a fast moving startup, you rarely get the chance to revisit problems so soon!

The biggest issue at hand with our previous Google flow was the way in which I’d chosen to re-authenticate users whose Wunderbar sessions had expired. In this case, OAuth provides a handy mechanism: a refresh token. Here’s how access tokens and refresh tokens are exchanged from the client and the authentication server in OAuth 2.0 flows:

At the time, we had session lifetimes of 60 minutes when accessing Wunderbar. These cookies also lasted 60 minutes. We faced a problem: if somebody walked away from their laptop for 61 minutes after authenticating, when they came back, how could we make it so they didn’t have to go back through the Google OAuth 2.0-initiated login flow from the beginning?

My solution at the time persisted a secure cookie with the refresh token value client-side with a longer expiration length. Then, if the user didn’t have a valid access token, we’d simply present the refresh token back to the server and get a new access token all over again. This worked well, but exposing secrets to the client always presents a risk of compromise. Refresh tokens are longer-lived, so they're of even higher value to protect!

Okta’s documentation made it very easy to stand up a simple authentication server and set up a local development server. Okta has a great blog post guide with tools we were already using (Rails and Omniauth, specifically) to get up and running. Beyond that, our biggest challenge was ensuring that we had a smooth rollout.

There was an issue in how we’d test and use this flow in our staging environments, however. At the time we were working on this (early 2020), there were two issues:

Our sandbox environment URLs were non-deterministic and could be varied depending upon how an engineer user named their cluster.

Okta apps did not support wildcard redirects in their oauth redirect URLs. This controls where, upon hitting Okta’s servers, they should redirect your payload to handle the meat of authentication.

For a time, our clever infrastructure engineering team managed around these issues with the use of an intermediary keycloak server. However, Okta now supports wildcard redirects that we’re successfully using today:

The biggest functional change to our authentication flow was in our refresh flow. The biggest challenge in landing this change was in how we deployed it to end users in production.

We didn’t want to expose the refresh token because it presented too much risk. What do we do instead? Something a little bit risky, but that we preferred after threat modeling our scenarios. Security engineering is often about making tradeoffs given a number of constraints. Today, our session lifetimes are down to fifteen minutes, and our refresh lifetime for an active session (e.g. somebody making at least one request within this window) is 60 minutes. If somebody comes back after their access token expires, that’s fine; we’ve persisted the encrypted refresh token to the database, and we can use attributes from the expired but still cryptographically valid token in order to look that refresh token up.

This was crucial for Okta especially because Okta refresh tokens are valid for 100 days by default!

Disruptive change is hard. We didn’t want to disrupt the workday of our hard-working ops users who are caring for our many customers and provider partners. The biggest intent here was to keep a safety blanket fallback to the old authentication flow until we were quite confident in its functionality. This broke down into a multi-phase approach:

1. Dark-launch the new Okta flow hidden behind a URL and with a small allow-list of users (mostly just the Security team) assigned to the new application in Okta.

2. Ask for volunteers! These volunteers would go about their normal work day in a variety of functional areas (customer care, pharmacy, fulfillment, engineering) using Okta instead of Google.

3. Change the default login flow to use Okta, but keep Google working just fine as a secondary action hidden behind a URL.

This allowed us to always ensure that we didn’t disrupt peoples’ days too much!

We achieved our goals and now, nearly two years later, we’re successfully using Okta’s OIDC flow within our core pharmacy management app. What did we learn in the process?

Not every API is necessarily fully implemented in every client. At the time of implementation, Okta’s Ruby client wasn’t fully supported. In order to support our refresh flow, we needed to upstream a small change into the client library, which required a little investigation.

Exposing secrets involves risk! In our case there wasn’t a glaring need to expose refresh tokens client-side at all.

Slow rollouts are great, especially with disruptive changes! A big thank-you to our team members who were willing to be part of the experiment group last year when this rolled out.

Interested in learning more about engineering at Alto? Follow us on LinkedIn and check out our open positions.

Overall heart health involves all the components of this complex network. Coronary artery disease — the most common type of heart disease — affects your blood vessels and the flow of blood to your heart. It typically develops over the course of many years, and many of the key risk factors can be managed. Here’s what to know about preventing and treating this type of heart disease.

Coronary Artery Disease, Explained

Your heart is at work every minute of the day, pumping blood to deliver oxygen to vital organs. Each part of the heart plays a role in this important task, along with the rest of your cardiovascular system, including your arteries and blood vessels.

Like any organ in your body, your heart needs oxygen to function. Coronary arteries are the blood vessels that supply your heart with oxygen-rich blood. Coronary artery disease occurs when these blood vessels are hardened or blocked, affecting the flow of blood to the heart itself.

This condition is often caused by atherosclerosis — the accumulation of substances like cholesterol and blood-clotting material in your arteries. (These substances are collectively referred to as plaque). A variety of factors can increase your risk for atherosclerosis — and therefore your risk for heart disease — including high LDL cholesterol, high blood pressure, diabetes, obesity, physical inactivity, and smoking. Additional key risk factors for heart disease include age and family history. 

There are several known symptoms of coronary artery disease, including chest pain or tightness (also called angina) and shortness of breath. Depending on where your blood vessels have narrowed, you may also experience pain, numbness, or weakness in your arms or legs. Women are more likely than men to experience additional symptoms such as nausea, back and jaw pain, and shortness of breath without accompanying chest pain.

Symptoms of coronary artery disease present differently from one person to another. It is best to discuss your risk of heart disease with your doctor if you have concerns rather than waiting for symptoms to appear. It’s especially important to be proactive if you have a family history of heart disease.

In the long term, coronary artery disease can increase your risk for other heart health issues, including heart attack and heart failure, which occurs when your heart cannot pump all of the blood that your body needs. These serious health risks are preventable — by identifying coronary artery disease at an early stage and following the treatment plan your doctor recommends, you can manage the condition and lower your risk for related complications.

Too often, people learn that they have coronary artery disease only after experiencing a more serious complication like a heart attack, stroke, or heart failure. Consulting with your doctor about your risk for heart disease and staying up to date with blood pressure, cholesterol, and blood glucose screenings is key to detecting heart health problems early on. Here are some of the numbers your doctor will track to identify your risk for heart disease, along with your individual and family health history.

The National Institutes of Health defines normal blood pressure for most adults as below 120 mm Hg systolic and 80 mm Hg diastolic. (Systolic pressure, the first number in your blood pressure reading, is typically the reading that’s most important when monitoring blood pressure.) If your blood pressure falls in this range, typical screening frequency is at least once per year. If your blood pressure is in a higher range, ask your doctor about recommended screening frequency.

Your cholesterol levels can also help your healthcare provider identify your risk for heart health issues. If your total cholesterol is under 200 mg/dL, you can have your cholesterol checked every 4-6 years. If you are at higher risk for heart disease and stroke, consult your doctor about recommended screening frequency.

In the long term, high blood sugar can damage your blood vessels, and diabetes is a risk factor for coronary artery disease. It’s important to monitor your blood glucose levels as part of your broader heart disease prevention efforts, especially once you enter your 50s and 60s. Your doctor can advise you on recommended screening frequency.

Remember that it’s never too early or too late to prioritize heart-healthy living. Read our previous blog, ‘How to Prevent Heart Disease at Any Age’ to learn about important heart disease prevention actions for your age group.

If you have coronary artery disease, your doctor may recommend lifestyle changes and/or medications to manage your heart disease risk factors. Quitting smoking, planning a heart-healthy diet, and exercising more frequently are often a first step in managing coronary artery disease.

The specific medications used to treat coronary artery disease vary by individual. Your doctor may prescribe one or more of the following types of medications depending on your health history:

Cholesterol-lowering medications like statins, fibrates, and niacin

Blood pressure medications like beta blockers, calcium channel blockers, and ACE inhibitors

Anticoagulants and antiplatelets to reduce your risk of blood clots

Heart-healthy living is a lifelong practice, and heart disease prevention or treatment is an ongoing commitment to your health. At Alto, we make it as simple as possible to manage your risk for heart health issues like coronary artery disease and stay on top of cholesterol-lowering or blood pressure management medications. 

We will work with your doctor, your insurance (if applicable), and any third party savings programs that you may qualify for to ensure your medications are as affordable as possible. And our team of pharmacists is available to chat whenever questions come up about side effects or how to take your medication properly.

Reach out any time via text or phone at 1-800-874-5881 or in-app messaging.

Last year, we set out to tackle this technical debt by splitting the User model into separate concerns and refactoring the entire system into this new paradigm.

Tackling Technical Debt and Improving Security: How Alto Untangled an Overloaded User Model

At Alto, we have three types of users: customers, healthcare providers (nurses, doctors, etc), and operations users who run the pharmacy. For each type of user, we maintain a separate application (though they share a monolithic backend for now): our customer app; our Alto Connect app for providers; and our internal pharmacy management tool called Wunderbar, which powers the pharmacy experience from prescription intake to fulfillment.

Early on, we made the pragmatic choice to represent both customers and operations users with a single User model, distinguishing them with a “roles” attribute. This required repetitive checks that, over time, slowed down our engineering teams, presented substantial privacy risk, and posed difficult scaling challenges as we evolved our monolith into different microservices. 

The challenges of relying on a single User model for different types of users

The notion of a User is core to our concept of identity from which authentication and authorization proceeds. From a security perspective, it’s difficult to build an extensible and clean authentication and authorization system when you’re overloading one model for two very different populations of users. From a privacy perspective, this muddied customer records (which are subject to more restrictions because they’re healthcare records) with workforce records, hamstringing many analytics efforts.

Authentication: Alto employees who were also Alto customers were required to use their work emails for their customer accounts. Not only was this restriction frustrating for Alto employees, but it also posed a security risk: employees should only be able to access Wunderbar via corporate SSO, not their personal account passwords. 

Authorization: Engineering new authorization capabilities required intricate knowledge of the idiosyncrasies of our user structure. This both slowed down development and created tremendous risk that any mistake could easily escalate user permissions.

Privacy: Because they existed in the same table, both customer records and Alto operations user records were classified as Protected Health Information, or PHI, which severely limited what we were able to do with them.

Attribution: Our robust audit logs were significantly less useful when a user ID in the transaction log might refer to either an ops user or a customer.

State management: Some updates pertaining to admin identity lived on the customer model, some lived on the employee model, and others lived in other various models.

Architecture: Tight coupling would have prevented us from decoupling our monolithic system into smaller services.

Our backend is primarily a monorepo Rails application that shares a tremendous amount of functionality between our customer, provider, and ops user-specific domains. Generally we use JWTs to secure customer and ops user sessions. These JWTs are issued only once valid credentials have been present.

Ops users are required to log in via Okta SSO, enforcing both valid username/password as well as a two-factor authentication (2FA) prompt. We use Devise and Omniauth to wire up the OIDC flow with our upstream identity provider, and we use CanCan to expand a set of roles per-user into a set of capabilities that restrict what resources they can access/modify.

Ops users primarily work in our internal pharmacy management app. However, the state of these users was shared between several different tables: Users, which otherwise refer to customers, and Employees, which share some ops user-specific state tied to Users.

Large change sets of code being deployed at once are harder to code review, harder to unwind if something goes wrong, and are more likely to be more disruptive. For this project, we knew that such a large architectural change had to be broken up into many more-digestible pieces if we were to be successful. The plan:

1. Create a new dedicated model for our ops users

# == Schema Information
#
# Table name: wunderbar_users
#
#  id                        :bigint(8)        not null, primary key
#  user_id                   :bigint(8)        indexed
#  email                     :string           not null, indexed
#  first_name                :string           not null
#  last_name                 :string           not null
#  token_last_invalidated_at :datetime
#  roles_mask                :integer          default(0), not null
#  title                     :string           not null
#  team                      :string           not null, indexed
#  is_active                 :boolean          default(TRUE), not null
#  slack_user_id             :string
#  pod                       :string

This was the easy part: create the database structure for a properly isolated admin user type, which we called a WunderbarUser.

We now had our admin user state split between Employees and Users, with a new table that engineers and analysts wanted to use that was not yet populated, and we had to make progress while allowing new hires and new customers alike to onboard to our platform.

2. Synchronize state between the old tables and new table

Next, we created a WunderbarUser for any User who was, or ever had been, a Wunderbar user. This required some historical digging —we had a lot of people who were onboarded, and even some offboarded, before we even had the intermediate Employee record!

Two major choices at this stage really helped the project succeed. First, we ensured that the WunderbarUser records all had IDs matching the corresponding User records. This is because a lot of our legacy audit records (that we’re required to preserve and keep useful as a Covered Entity under HIPAA) referenced an `admin_user_id` column populated with the ID of the User. If we just created all new entries for WunderbarUsers, these old logs would require at least one level of indirection (e.g. querying back to User) to make useful.

     # Create with the same ID as the user that we're creating this from.
     wunderbar_user_attributes = {
       id: user.id,
       user_id: user.id,
       email: user.email,
       roles_mask: user.roles_mask,
       date_of_birth: user.date_of_birth,
       token_last_invalidated_at: user.token_last_invalidated_at,
       first_name: user.first_name,
       last_name: user.last_name,
       preferred_first_name: user.preferred_first_name,
       is_active: false,
     }

Second, we created a new WunderbarUser any time we created a new Employee record (e.g. when we onboarded a new employee) and synchronized any changes between Users, Employees, and WunderbarUsers tables. As a side note: we knew we’d have to protect this syncing logic to ensure that we only copied each change once — otherwise, our callbacks would fire forever!

3. Migrate functionality slowly to use the new ops user model

We migrated logic to reference WunderbarUsers where previously we’d referenced Users and/or Employees. Authentication, authorization, audit logs, and pharmacist licenses were all updated here — albeit painstakingly!

We then migrated all of the foreign keys our various Postgres database tables held to either the Users or Employees tables to instead reference the WunderbarUsers. This ended up being around a hundred foreign keys, but a constraint-only migration that fortunately didn’t involve any data changes.

In short: our goal was to do this change slowly and steadily enough that nobody would notice until, one day, we could announce to the team, “Hey, we did a big security-pertinent project, and now you can change your customer email to be anything.”

Authentication: While we use the same JWT format for each, our customer and operations user authentication systems are now totally separated.

Authorization: With separate user types, we have separate capabilities and domain-specific authorization checks for each area, enforced in code.

Experience: Onboarding as a new Altoid (somebody who works here; come join us!) is totally separate from the customer experience, and you no longer need to link your accounts together.

Privacy: Our ops users who choose to be customers enjoy the full privacy afforded to our customers without tying their account to their work identity in any way.

Attribution: Legacy audit logs are still index-able and query-able with the same IDs, just in a slightly different table.

State management: Customer state lives on the User table, and ops user state lives on the new WunderbarUsers table. Our callbacks on each have been dramatically pared down, and our intermediary Employee table is entirely gone, along with a number of other now-ancillary tables.

Architecture: We have a clear path to federate RBAC checks without shipping our entire customer database between services.

Bonus: Performance and code readability generally improved by removing complex joins and join tables and isolating concerns between the customer and ops user domains.

Another Bonus: Significant refactorings and increased test coverage along the way while we cleaned up some very old, very stale code.

Large architectural changes of legacy systems with heavy use are rarely 100% smooth. Here are a few learnings to keep in mind the next time you tackle a hefty piece of technical debt. 

1. Always validate your assumptions (e.g. the data your constraints are against) when foreign keys are concerned. Multiple times we assumed, “Hey, we’ve migrated a bunch of really big tables that are really old (insurance claims, prescriptions, and more), so there’s no way we could have another admin user that we haven’t created a commensurate WunderbarUser for, right?” Wrong! Your data can always have inconsistencies, but a simple SQL query (e.g. validate that all values of the column you’re migrating the foreign key to have a matching WunderbarUser) can save a lot of time and headaches.

2. When migrating foreign key constraints on tables with active reads in Postgres, ensure you break it into two steps. Migrate your foreign key with `{ validate: false }`

 def up
   remove_foreign_key :inventory_orders, to_table: :users, column: :created_by_id
   add_foreign_key :inventory_orders, :wunderbar_users, column: :created_by_id, validate: false
 
   validate_constraint
 end

Validate the named constraint of your foreign key separately

 def validate_constraint
   validate_constraint_statement = 'ALTER TABLE "inventory_orders" VALIDATE CONSTRAINT "fk_rails_abcd1234";'
   ActiveRecord::Base.connection.execute(validate_constraint_statement)
 end

3. When working with legacy tables, be mindful of legacy systems. Our Billing service used a very outdated golang ORM that panicked if a column specified by the `struct` in source no longer existed; this caused a late-night outage that made auto billing requests back up the next day.

4. Scoping projects is hard! This was a larger project than our team had ever scoped and delivered before. Once all the dust settled, this project took several months longer than our initial scoping estimates.This project was worth doing, but better estimates allow us to prioritize better. If we’d had a great scope of this immense project upfront, would we have chosen to do it first? Also, this was a tremendously difficult project to separate into material stages; all the benefit was really backloaded.

In the end, this was a very satisfying and important project to complete. We wouldn’t have been able to accomplish it without a lot of help from our colleagues on the Platform team, the Security team, and across many other functions atAlto. Very large projects that involve a lot of moving pieces and impact other teams are always scary — but we can’t let that fear prevent us from making the changes we know we need to do in order to be successful long-term.

The Engineering Team is hiring! Learn more about open positions here.

Emphysema — which falls under the umbrella of chronic obstructive pulmonary disease (COPD), along with chronic bronchitis — is a lung disease that most often develops as the result of many years of heavy smoking. It affects more than 3 million people in the United States.

Emphysema: Causes, Symptoms and Treatments

As a progressive lung disease, symptoms of emphysema become more pronounced over time. While there is no cure, emphysema is both manageable and preventable. You can slow the progression of the disease by following the treatment plan your doctor recommends and maintaining a healthy lifestyle.

Read on to learn more about the causes and symptoms of emphysema and common treatment options.

Emphysema involves slow and gradual damage to the tiny air sacs in your lungs (alveoli). As these air sacs weaken over time, the surface area of the lungs is reduced, limiting how much oxygen can travel to your bloodstream.

Since emphysema progresses gradually, signs and symptoms may not appear until the lung tissue has already been significantly weakened. For many people, the first noticeable symptoms include shortness of breath and more frequent coughing, especially during physical activities. Other symptoms of emphysema include wheezing, exhaustion, and chest tightness, and producing greater amounts of mucus.

In some cases, the lack of oxygen can cause blue or gray lips or fingernails. If you experience this symptom, seek immediate medical attention.

The majority of emphysema cases occur as the result of cigarette smoking, with heavier smoking linked to a higher risk of developing emphysema. Additional risk factors for emphysema include the following:

Age — in many cases of smoking-related emphysema, symptoms first become noticeable between the ages of 40 and 60

Exposure to high pollution chemical fumes or lung irritants

Having a history of childhood respiratory infections

To determine if you have emphysema, your doctor will ask about your health history and the potential symptoms of emphysema that you are experiencing. They may ask you questions about your smoking history, your exposure to secondhand smoke or chemical fumes, and if you have a family history of lung health issues.

After taking your health history, they may order one or more of the following tests to confirm a diagnosis of emphysema:

Imaging tests, including chest X-rays and computerized tomography (CT) scans, which combines X-ray images taken from different angles.

An arterial blood gas test, which measures how well your lungs transfer oxygen to your bloodstream and remove carbon dioxide.

An electrocardiogram (ECG) to check your heart’s function and rule out a diagnosis of heart disease.

A pulse oximetry, which measures the oxygen level of your blood. This can show how well oxygen travels to the parts of your body furthest from your heart.

Lung function tests, which measure how much air your lungs can hold, how easy it is for them to empty air, and how well they deliver oxygen into your bloodstream.

Quitting smoking is the most important thing you can do to slow the progression of emphysema. If you are having trouble quitting, ask your doctor for smoking cessation resources like nicotine replacement therapy to help manage withdrawal symptoms and cravings. In addition, the treatment options below can slow the progression of emphysema and help you live an active lifestyle.

Several types of medications are used to manage and treat emphysema.

Bronchodilators like albuterol (Ventolin®, Proventil®, AccuNeb®), albuterol sulfate (ProAir® HFA®, ProAir RespiClick), and levalbuterol (Xopenex®) help relax your airway muscles and open your lungs to reduce discomfort. Although not common, potential side effects of bronchodilators include nervousness or shakiness, headache, throat or nasal irritation, and muscle aches. Your doctor can explain more about the risks and benefits of using a bronchodilator when building your treatment plan.

Blog

Diabetes Resource Round-Up

How We Used Kafka to Build an Event-Driven Automation Pipeline and Scale Our Pharmacy

Developing Robust Authentication Using Rails, Omniauth, and Okta

Coronary Artery Disease, Explained

Tackling Technical Debt and Improving Security: How Alto Untangled an Overloaded User Model

Emphysema: Causes, Symptoms and Treatments

Considerations and Approaches for Implementing Biometric Login: Part 2

Considerations and Approaches for Implementing Biometric Login: Part 1

All About HIV: Stages, Symptoms and Treatment

Top Questions to Ask Your Fertility Clinic

Celebrating National ERG Day at Alto

How Sleep Affects Heart Health & Tips for Better Sleep

Switch to a better pharmacy.