5 Keys to a Modern Data Architecture

Andrew Stewart

In recent years, modern data architecture has been an increasingly common topic when I meet with clients. Companies across all industries are realizing the value of analytics and want to make sure they’re able to fully leverage their data. To best address this subject, I find it important to focus on the desired business outcomes instead of focusing solely on the architecture itself. Based on my experience, I’ve found the following principles to be critical to the success of an enterprise data program.

1. Get the Right Data to the Right People at the Right Time

Data is only useful if people can act on the data in a timely manner. It must be accurate and actionable. However, “people” in this case means several different things. No matter if we’re talking about external people (e.g., customers, etc.) or internal teams (e.g., marketing, operations, etc.), data must be easily consumed, visual, and simplified. Therefore, data needs to be delivered in the context of the persona and relevant for the individual.

When starting with a persona-based approach, it is critical to build your next generation data platform by focusing on the people you’re looking to serve. “Build it and they will come” isn’t a good strategy when it comes to data platforms, unless you’re highly in-tune with the end-user’s needs, and you have a way to mandate the tools they use. But, if you’re like most companies (and statistically speaking you are), we’ve found that people resist change and there is a science to tackling adoption challenges. An easy way to foster success in this area is including end-users in your project team early on and seeking their input often.

2. Develop a Flexible Architecture

Businesses are always changing and data architectures are notoriously inflexible, especially in a highly relational data model. Simply put, data is rigid. However, you can craft your architecture to allow for flexibility of the data types you ingest and the ways you deliver information to each persona.

That being said, you will eventually model your data. Whether you use schema-on-read or schema-on-write, there still is a schema at some point. Here are a few things to consider when thinking about your architecture with flexibility in mind:

Limit the number of transformations of the data (i.e., simplify the layers within the data processing).
Look at opportunities to virtualize the business’ view of the data and reduce how many times you physicalize the data model(s).
Use high-performance database tiers to eliminate the need for “Cubes”, summary tables, or other performance techniques that were popular 10-15 years ago or more.
Consider a data sandbox (or refinery) as an area to give business analysts access to large amounts of data and use this to prioritize your data backlog.
Leverage event-based data streams (e.g., Kafka, etc.) to move data to the right places quickly.

3. Scale Up (and Down) Quickly

This one is simple: Don’t be afraid of the cloud. There are several models out there that will show you the cloud is cheaper, more flexible, faster to scale up and scale down, and more secure. Additionally, cloud technologies can be used to quickly prove the value of new data sources and help prioritize what goes into the new data architecture (not all data should be put in the data platform).

So what else do you need to know about the cloud? Be great at your data and don’t worry about being great at your infrastructure.

4. Commit to Security from the Beginning

Security is critical, and it should be a topic discussed in the foreground of the project. To avoid problems down the road, design for your security needs from the beginning.

By using a persona-based approach, you can create security requirements in the early phases of development that meet the needs of all users. Security can then be applied to the raw data instead of an ad hoc network of data sets and restrictions in the data presentation layer.

5. Account for Machine Learning

With the emergence of technologies like DataRobot and Azure Machine Learning Studio, it is becoming easier than ever to incorporate machine learning (ML). Even if the thought of ML is intimidating right now, it is important to create an enterprise data program that will allow your business to leverage predictive analytics solutions when you’re ready.

To do this, it is important that your data is clean and well organized. Potential ML tools should be evaluated and their requirements should be considered when developing your architecture. When the time comes, this will enable you to find key patterns in your data with minimal changes to your current architecture.

Modern Data Architecture for Your Company

Given the importance of data in today’s market, it is critical to make smart decisions when investing in a modern data architecture. While implementations may vary from business to business, I have found these principles to be consistent for successful projects.

If your company is wondering how to put these principles into practice, feel free reach out to us at findoutmore@credera.com. We’d love to help as you think through how your company can build an enterprise data program that sets you up for long-term success.