Using Pebble Templates in WhereScape RED to Deal with Hard Deletes in an ODS Table

Using Pebble Templates in WhereScape RED to Deal with Hard Deletes in an ODS Table

Using Pebble Templates in WhereScape RED to Deal with Hard Deletes in an ODS Table. 


 

In a recent YouTube video, we discussed how to use Pebble Templates in WhereScape RED to Deal with hard Deletes in an ODS Table

Giving an overview of WhereScape RED, and the benefits it has for you and your organisation.

Then delving into Data Stores and how we expect them to work, especially around Historic Data Stores.

Enabling you to store data and capture changes to your data in a historic Data Store, WhereScape RED is a great piece of software to do this.

Also, we discussed how we have created our FREE Pebble Template which can be run as a custom procedure after loading the Data into the Data Store.

Our Pebble Template has been designed to identify and end or update the DSS_CURRENT_FLAG and consequently update the DSS_END_DATE in line with the setting within the Data Store.

To find out more, watch the video:

10 Tips for Making a Data Strategy Work for You

10 Tips for Making a Data Strategy Work for You

Data Strategy is a complex subject, but one that can make all the difference in your business. Whether you want to generate more sales or improve customer support, you can do several things to get the most out of your data.

Here are our 10 Tips and Tricks for making a Data Strategy work for you!


Company Objectives AKA Top-Down:


1. Understand Business Goals

Knowing your company’s direction is essential before you start implementing a Data Strategy. You could have all the data in the world, but if it isn’t used correctly, what good is it?

Tip: You will often hear that data strategy is the ‘new business strategy’. And as such, you need to understand and analyse the information within your organisation to make informed decisions. You can do this by developing a data strategy that includes goals, issues and drivers for collecting data, as well as considering what types of data exist within your organisation – and where those sources may have come from.


2. Have a Data Strategy

You’re just wandering in the dark without a plan! Your employees must know about the importance of data and how it affects their work. By building a culture where data is valued, you can leverage its power to make better business decisions

Tip: Ensure that you have top-down buy-in from the top level of the company and that the Data Strategy is linked to business objectives. It will ensure that crucial business members are invested in the success of the data strategy if it means that the business objectives are successful!

People and Culture:


3. Build a Data-Driven Culture

You can’t manage what you don’t measure! your employees must know the importance of data and how it affects their work. By building a culture where Data is valued, you can leverage its power to make better business decisions.

Tip: it might not be worth collecting the data if you can’t answer the question, “what does it mean for me?”. Instead, think about what business problem you want to solve through the data, what do you need to know, or what you want to achieve.

Tip: Empower your team to make decisions based on data; you’ll be able to achieve the best results possible. The best way of doing this is by sharing information and allowing them to ask questions.

Tip: You can manage what you don’t measure! you must use the available data to make decisions about your company. Data can help you understand where there are gaps in some regions of your company, such as Sales and Marketing, and how they can be filled with more effective strategies.


4. Know your Data

It’s not enough to have data; you also need to know what it means. Look for patterns and trends in your data that can help you identify areas of opportunity and potential problems or issues that may pop up.

Tip: The key to success in data strategy lies in knowing what questions you are trying to answer and then identifying the ideal data required to answer those questions. Remember that nig data does not always mean the best data. It is important to think small. For example: by first understanding what questions your organisation whats to ask before working out what information you need to obtain.

Tip: It might not be worth collecting the data if you can answer the question, “what does it mean for me?” Instead, think about what business problem you want to solve through the data. What do you need to know, or what do you want to achieve?


5. Use Experts

Data is essential and can be highly confusing. From understanding how the data is created as part of the business process to sourcing data from the correct place – if you don’t have all the time or resources to devote to this, hire an expert who does! They will help you understand what your data means and how to make it work for you.

Tip: Understand the current data skills within your organisation. Will this enable or hinder your data strategy?

Tip: Can you train people to reduce the skill gap?


Process:


6. Data Governance

Data Governance is critical to data strategy success. Without proper leadership and policies that support information governance, your organisation can find itself overwhelmed by the complexity of managing data across its business units.

Tip: Start understanding who owns the data = from the moment it is created. These data owners know more about the data and how it can be properly used.


7. Change Management and Implementation

Data Strategy success hinges on change management. This is often overlooked in Data Strategy but is critical to a data strategy’s success. Change management has two parts:

– Ensure employees know why they should be excited about the new data strategy and how it will help them.

– Changing how things are done, so the new system works better after implementation.

Tip: There are always two sides to change, business and Technology. Make sure you coordinate both to avoid creating problems further down the line.


Technology:


8. Make Data Accessible

You don’t know anything without clear metrics! Your employees must know about the importance of data and how it affects their work. By building a culture where data is valued, you can leverage its power to make better business decisions.

Tip: Ultimately, reporting data aims to help you make informed decisions. Data Visualisations and presentations play an essential role in ensuring that the key insights from that data aren’t presented to the wrong people in the wrong way. At this stage, keeping your target audience in mind is perhaps one of the most important things to remember.

Tip: Once you understand what data is needed, how it will be turned into value, and how it will be communicated to the end user, there are several software and hardware considerations that you will need to address. What current analytic and reporting capabilities do you have? Should your legacy systems be supplemented with cloud solutions? How will the final reporting platform look when it’s deployed?


9. Using Technology:

You may not be able to buy all the technology you want on day one. Think about where your data is stored, processed and used. Different businesses have different concerns with processing efficiency and cost-saving storage. Others will have no option but to use the technology they already have.

Tip: Only collect, store and process the data needed for the data strategy’s primary objectives. Once successful, you can increase the scope to manage the remaining data.

Tip: Remove duplications as quickly as possible. If anything, this is a cost-saving exercise.


10. Keeping Up with the Times

Technology trends can significantly speed up a company’s data strategy. In recent years, due to COVID and the increased mobile nature of our daily lives, cloud technology and platforms have become more accessible. Data visualisation tools have also been accessed on more devices and device types.

Tip: Sign up for technology newsletters to understand the products’ development and roadmaps.


Your Data Strategy will not succeed if you do not have a plan to execute it.

This step-by-step guide will help you develop and implement your data strategy by identifying the key stakeholders, defining your KPIs and laying out the first steps for making things happen.

By working with the Data Professionals at Engaging Data, we will work with you to implement everything mentioned above and make your Data Strategy effective to make better business decisions and leverage the true power of data for your organisation’s longevity.

The Problems Documenting a Data Warehouse

The Problems Documenting a Data Warehouse

The Problem Documenting a Data Warehouse

More data is being collected, stored, and analysed than ever before. One of the digital age challenges is how and where we store all this data safely and accessibly.

A modern Data Warehouse can solve many of these issues, using multi-tiered architecture to ensure different users with various needs can access the information they need. In order to expand and develop a Data Warehouse, documentation is invaluable.

Are you considering approaching a Data Warehouse using a documentation method? Then read on to find out more!

What is Documentation?

Data documentation is vital in many ways for a Data Warehouse, and it’s how you can ensure that your data will be understood and accessible by any user across your organisation. Documentation will explain how your data was created, its context, structure, content, and any data manipulations.

Documentation is crucial if you’re looking to continue developing, expanding, and enhancing your Data Warehouse. However, it’s essential to understand what documentation entails to ensure your Data Warehouse operates smoothly and its processes run smoothly.

Documenting a Data Warehouse

Like we said, the amount of data that we collect as store as organisations is increasing and traditional Data Warehousing that may be set up using a simpler database structure will often struggle to cope. Partially with the sheer volume of information it needs to store and analyse, it also needs to be accessed by various users, often in different ways. A document-based approach to data warehousing will allow for streamlining of data from multiple sources and multi-user access.

When documenting your Data Warehouse, you should begin with creating standards for your documentation, data structure names, and ETL processes, as this creates the foundation upon everything else is built. A robust and excellent Data Warehouse will have straightforward and understandable documentation.

A successful Data Warehouse implementation will often come down to the data solution’s documentation, design, and performance. However, if you can accurately capture the business requirements, then using documentation, you should be able to develop a solution that will meet the needs of all users across an organisation.

At Engaging Data, documenting a Data Warehouse has become second nature. Although it’s not necessarily the easiest or most logistically straightforward part of the process, it’s necessary to ensure your data warehouse processes run smoothly.

What Documentation do I need for a Data Warehouse project?

The exact pieces of documentation that you need may vary by your particular Data Warehousing project. However, these are some of the significant elements of documentation that you should have:

The Business Requirements Document

will outline and define the project scope and top-level objects from the perspective of the management team and project managers.

Functional/information requirements document

which will outline the functions that different users must be able to complete at the end of the project. This document will help you to focus on what the Data Warehouse is being used for and what different pieces of data and information the users will require from the data warehouse.

The fact/qualifier matrix

is a powerful tool that will help the team understand and associate the metrics with what’s outlined in the business requirements document.

A data model

is a visual representation of the data structures held within the Data Warehouse. A data model is a valuable visual aid to ensure that the business’s data, analytical and reporting needs are captured within the project. Plus, data models are helpful for DBAs to create the different data structures to house the data.

A data dictionary

is a comprehensive list of the various data elements found in the data model, their definition, source database name, table name and field name from which the data element was created.

Source to target ETL mapping document

which is a list focusing on the target data structure, plus defines the source of the data and any transformation that the source element goes through before landing in the target table.

What are the problems of Documenting a Data Warehouse?

Documenting a Data Warehouse can be a massive project, depending on the amount of data, the number of users that need access, and the business requirements. As the amount of data held within a Data Warehouse increases, management systems will need to dig further to find and analyse the data. This is especially an issue within traditional Data Warehouses, and as data volume increases, the speed and efficiency of a data warehouse can decrease.

Generally, spending time to understand and document your business needs will make documenting your Data Warehouse easier because Data Warehousing is driven by the data you provide. If you don’t take the time to map these critical pieces of information early in the process, you may run into problems later on. Similarly, the correct processing of your data and structuring it in a way that makes sense for your organisation today and in the future. If you don’t set yourself up for the future, structuring data becomes more complex and can slow down the processing as you add more information to your Data Warehouse. In addition, it can make it more difficult for the system manager to read the data and optimise it for analytics.

Overall, the better the initial documentation, planning, and business information model are, the easier your implementation process will be and make it easier to continue to add data to your warehouse. By carefully designing and configuring your data from the start, you’ll be rewarded with better results.

Another potential problem in documenting a Data Warehouse is choosing the wrong warehouse type for your business needs and resources. Many organisations will allow various departments to access the system, stressing the system and impacting efficiency. By choosing the right type of warehouse for your organisation and making a future-proofed decision, you can balance the usefulness and performance of your data warehouse.

Data Warehousing is an excellent system for keeping up with your business’s various data needs. By making many long-term decisions and preparing at the start, you can avoid many potential problems when documenting your data warehouse. However, you can prevent many challenges associated with data warehouse deployment and implementation by utilising a tool like WS Doc.

What is WS Doc?

WS Doc is a simple-to-use tool that automates a lot of the processes of documenting your data warehouse by automating the publication of WhereScape documentation to your choice of WIKI technology.

In addition, with WS Doc, you can collaborate on workflows, editing data sets and input, allowing various users to work on the project simultaneously. As well as integrating with other apps and systems, WS Doc makes collaboration and streamlined working possible.

Why was WS Doc created?

WS Doc sought to bring document automation and assembly to more industries, turning tedious and detailed work into automated processes and systems.

By allowing you to gather data and instantly generate template documents, even generating document sets from your data, you can save up to 90% of the time that you’d have spent on drafting documentation.

By automating the publication of WhereScape documentation to your choice of WIKI Technology (Confluence, SharePoint, GitHub, or something else), you’re providing your documentation with the power of the WIKI technology, allowing it to be easier to digest, apply, and share.

Overall, WS Doc streamlines and automates the process, speeding it up and making it less resource-heavy.

 

Want to learn more about WS Doc?

Click the button below. Everyone is on the same page with WS Doc.

Why Choose WS Doc?

In conclusion, by choosing WS Doc to document your Data Warehouse project, you’re utilising a simple tool to automate processes that otherwise would take a long time, as well as using a lot of resources, and that’s not even considering the possibility of human error in a process that requires a lot of detail and repetitive actions.

We’ve discussed some potential problems you can run into when documenting a Data Warehouse. However, with WS Doc you can overcome these issues because WS Docs is a tool that promotes effective communication and collaboration, engaging with the people using data. It saves time and resources by automating the publication and implementation of documentation. And finally, it ultimately enhances your existing toolset, offering a developed, streamlined, and simple-to-use experience.

 

Here’s at Engaging Data, we use WS Doc in the documentation of Data Warehouse projects we carry out for our clients.

If you’d want to learn more about the process or see if WS Doc could be the right tool for your organisation, schedule a call with us!

How is CI/CD (Continuous Integration / Continuous Delivery) Used to Modernise a Data Warehouse? 

How is CI/CD (Continuous Integration / Continuous Delivery) Used to Modernise a Data Warehouse? 



.

How is CI/CD (Continuous Integration / Continuous Delivery) Used to Modernise a Data Warehouse? 


A specially designed type of data management, a data warehouse is a system that has been built to enable and support business intelligence and analytics. For example, a data warehouse will read large amounts of data to understand relationships and trends within an organisation.  

Data warehouses contain large amounts of historical data and are intended to perform queries and carry out analysis. The data within a data warehouse is often derived from various sources, such as application log files and transaction details. You’re centralising and consolidating large amounts of data into one location by utilising a data warehouse. Over time, this data can be invaluable to data scientists and business analysts, allowing informed business decisions to be made using valuable business data insights. As this data record builds, a data warehouse is often considered an organisation’s single source of truth.  

Analytics and data have become indispensable to allow businesses to stay competitive, and companies rely on reports, dashboards, and analytic tools to extract data, monitor business performance, and support future business decisions. The power behind these processes are data warehouses, which store data efficiently to minimise the input and output of data and deliver query results quickly.  


What’s the Architecture of a Data Warehouse?  

A data warehouse architecture is created in tiers. The top tier is the front-end, where results are presented through reporting, analysis, and data mining. The middle tier is where the analytics engine used to access and analyse data sits. The bottom tier of the architecture is the database server, where data is loaded and stored.  

Data is stored in two types of ways:  

  1. Data accessed regularly is stored in fast storage, like an SSD drive.  
  2. Whilst data that is less frequently accessed is stored in an object store.  

The data warehouse is competent at differentiating between the two types of data. It will automatically ensure that frequently accessed data is available in the fast storage option, thereby optimising query speed.  

So, how does a Data Warehouse work?

A data warehouse might contain multiple databases, and within each database, the data will be organised into tables and columns. Within each column, you can then define a description of the data, for example, integer, string, or data field. Tables can be organised within schemes, using a similar structure to files and folders. So, data is added to a data warehouse, it’s then collected and stored into various tables defined by the schema, and query tools utilise the schema to identify which data tables to access and analyse.  

What are the benefits of using a Data Warehouse?

As we mentioned above, there are multiple benefits to using a data warehouse, for example, making more informed business decisions based on data and insight. Here are some of the additional benefits of using a data warehouse:  

  • Consolidated data from many sources 
  • Historical data analysis 
  • Data quality, consistency, and accuracy 
  • Separation of analytics processing from transactional databases and improves the performance of both systems 

What’s a Modern Data Warehouse?

Across your organisation, multiple teams and users will have different needs for a data warehouse. Traditional data warehousing can’t always keep up with the demands of rapidly growing volumes of data, processing workloads and analysing data. In contrast, a modern data warehouse architecture addresses the different needs of your organisation by providing a way to manage everything you need with various integrated components that all work together. 

A modern data warehouse would typically include:  

  • A streamlined database that simplifies the management of all data types and provides different ways to use data across your organisation 
  • Self-service data ingestion and transformation services  
  • Support for SQL, machine learning, graph, and spatial processing 
  • Multiple analytics options that make it easy to use data without moving it 
  • Automated management for simple provisioning, scaling, and administration 

Ultimately, a modern data warehouse can efficiently streamline data workflows in a way that traditional data warehouses can’t. This allows everyone to perform their jobs more effectively and efficiently, from analysts and data engineers to IT teams and data scientists.  

If you have a lot of data, and multiple teams that all need to access it, then modernising your data warehouse should be a key component of your data strategy. But how can you update your data warehouse? This is where CI/CD, which stands for Continuous Integration (CI) and Continuous Delivery (CD), comes into play. CI/CD creates a faster, more precise, and overall efficient way of combining the work of multiple teams into one streamlined product. For example, in app development and operations (DevOps), CI/CD would streamline coding, testing, and deployment by creating a single space for storing work and tools, thereby consistently combining and testing code to ensure it works.  


What is the CI/CD pipeline, and how can it support modernising a Data Warehouse?  

By utilising a CI/CD pipeline, any software development or engineering processes that combine automated code building with testing and deployment, you can deploy new and updated software safely and precisely.  

In other words, a CI/CD pipeline is the behind-the-scenes plumbing of your data and analytics, making your life easier and your work more consistent. Here at Engaging Data, we have developed a solution that integrates WhereScape 3D & RED with CI/CD and DevOps pipelines in one ecosystem to modernise the world data warehousing. If you’d like to know more about our CI/CD pipeline, take a look here.  

Because a CI/CD pipeline isn’t just a linear process, it allows DevOps teams to write code, integrate it, test, deliver updates & releases, and make changes to the software in real time. In addition, the ability to automate critical parts of the CI/CD pipeline allows development teams to work more efficiently and more effectively and improve other DevOps metrics.  

What are the benefits of the CI/CD pipeline in modernising a Data Warehouse? 

The most significant benefit of a CI/CD pipeline is the automation of releases from the initial testing to deployment. Additional benefits of the CI/CD pipeline for DevOps include:  

  • Automated testing makes the development time more efficient; CD and automation mean that a developer’s changes to a cloud application could go live within minutes.  
  • Thanks to faster, more efficient testing and development, less time is needed to be spent in the development phase, therefore reducing cost.  
  • The CI/CD pipeline is a continuous code, test, and deploy cycle. Every time code is tested, developers can react to feedback and improve the code.  
  • A CI/CD pipeline allows a more collaborative and integrated process with everyone across the organisation who needs to access the data warehouse.   

If you have any questions about a CI/CD pipeline or the deployment process, we have diagrams of the flow here.  


Conclusion

Overall, a modern data warehouse utilising CI/CD should be an essential part of your data strategy if your organisation has multiple touchpoints requiring access to data, insight, and analytics.  

Do you have any questions about modernising your data warehouse? Or about creating and implementing a CI/CD pipeline? Our expert team of data specialists can implement a modern data warehouse for your organisation.

Fill out our form below, and one of our team will be in touch.  


The Gold Standard – A Comprehensive Guide

The Gold Standard – A Comprehensive Guide

Engaging Data Explains :

The Gold Standards in Data –

A Comprehensive Guide


Achieving an excellent level of data architecture is far from easy. But it is certainly possible if you implement certain guiding principles. Central to this is the implementation of a gold standard benchmark, which can then underpin any effective data architecture operation.

However, a gold standard is not something that comes naturally. In our experience, it requires diligent thought, effort and openness to change.

Within this blog post we’re going to discuss some of the considerations related to this important goal for many organisations.


Resource Considerations

When we think about creating engaging analytics or data platforms to shape the growth of an organisation, the focus is often on the finding tool capable of developing the solution and not the surrounding aspects. But all of the ingredients that go into the mix are critically important.

Imagine you’re the owner of a cake shop providing bespoke cakes for your customers. Your products have to be good enough to keep customers coming back, but they also have to retail at an attractive price point. This means that there are immediately resource considerations.

You may choose to focus on providing a premium product, creating high-quality goods for a premium price.  Alternatively, you may deliver a higher volume product, baking lots of different cakes on a larger scale, which are still of a good standard, but only suitable for a lower price point.

In order to make this decision, you need to understand the following:

  • Product – what we are providing, and the value that we create.
  • Place – the environment that makes it possible to create the necessary standard of products at a sustainable rate.
  • People and Process – the team, the processes and the delivery environment (the bakery, the storage, the front of house, stock control, delivery, billing, etc.) that produce and maintain the consistent quality of product and experience.

If you can put all this together then you have the beginning of a gold standard in cake production. By the same token, in our field Engaging Data helps companies to review all of the data elements supporting such a setup, combining this with their aspirations to form bespoke gold standards. This enables our clients to achieve profitability and success.

Controlling The Input

Controlling input is a critical component of processing gold standard products. These vary, depending on what you are trying to produce, but examples include:

  • Requirement gathering.
  • Sources of data.
  • Quality of data.

Controlling inputs and creating quality is critically important, as if you put terrible into a system then the ultimate outcome will be a terrible product! Thus, you need to understand the requirements of your customers. In the cake shop example, this would mean knowing what type of cakes your customers desire, the toppings needed, the date and time of delivery, any dietary requirements, and so on.

The good news is that controls can be quite straightforward. They can be something as simple as checking data. So in a cake shop, it’s vital to confirm the direct requirements of your customers, noting down all relevant information. This can make the vital difference between providing the ideal products for your customers, or producing something that seems excellent, but is rendered useless or sub-par by one important constituent. For example, you might produce a cake for someone with allergy needs that is simply inedible from their perspective.

Quality control can be achieved by creating a simple order form. For example, a cake shop might include:

  • All vital information being distilled into yes/no questions – eg. “should cake contain nuts?”.
  • Ensuring that all product types are selected, and that nothing out of the unusual is ordered.
  • Product limitations being noted expressly on the form – acting as a reminder and preventing incorrect ordering.

Such a review process ensures that information is gathered correctly, and creates a collective responsibility for discerning the appropriate information. Important questions that you can ask yourself in a data environment to acquire such critical information include the following:

  • How will the requirements come into the team? 
  • How do we need to record them?
  • Do we have the right tools to collect the data?
  • How will we handle data quality?

Output Consistency

The output is the result of your efforts, so you have an innate interest in ensuring that it’s the best possible product. In common with the input, it is important to understand what you can control to reduce risk, as this can have a big impact on your output.

Central to this process is building systems and controls that enable you to monitor outputs. This in turn makes it possible to assess if they need to be altered in any way. This means that in a cake shop, you may consider the impact that each of the following areas has on the supply chain of cakes:

  • Production Team (bakers, shop front, etc.).
  • Ensuring similar standards and experience.
  • Providing the same customer experience.
  • Ensuring knowledgeability about the production processes, industry and competitors.

Each aspect of the order and production process also needs to be assessed and standardised:

  • Enjoyable and consistent ordering experience.
  • Stock control to manage high-quality ingredients. 
  • Quality control of all products.
  • Meeting all food hygiene regulations with a 5-star rating.

And then the tools of the trade should also be taken into consideration, as part of an ongoing auditing process. Central to this is ensuring that any equipment being used is within acceptable operational parameters, particularly not being overloaded or overstretched in any way.

So when you’re working in a data environment, or any working context, if you want to create gold standards then it’s important to continually monitor and challenge your processes. Ask yourself questions continually, such as:

  • Do we have the right team in place?
  • Do we understand what standard of products that we need to create?
  • Do we have processes in place that enable us to produce quality products?

Reviewing is never the most exciting nor relished procedure! It often feels like an unnecessary grind that slows down your whole operation. But it’s actually a critical part of any good business, enabling you to identify major faults before they become ingrained.

Reviewing Processes

We’ve already seen in the first part of our gold standards blog post that the cake shop reviewed their existing processes before making a decision to change their order form. Often it’s only from self-assessing in this way that you can uncover new ways of working that enhance your productivity and efficiency.

It’s also important to emphasise that this needs to happen across the business. It’s normal to have a review as part of project governance for new projects, but it’s always worthwhile to reassess existing projects as well. They may be ‘good enough’, but they also might not be reaching the gold standard.

So in our cake shop scenario, how will the business assess the standard of the cakes being produced? Well, when the business reviewed the kitchen they found that each cook bakes one order, while also being responsible for checking their own cakes before they move to the decoration stage. This all seemed fine, but then the retailer received a few complaints about burnt edges and sub-par ingredients. Consequently, the cake shop reflected that its existing process of ‘marking your own homework’ was not sufficiently robust to identify problems.

In order to address this issue, several ways of reviewing the production of cakes were decided upon:

  • A simple visual inspection – does the cake look uncooked or overcooked?
  • A thorough test, such as breaking the centre of the cake to see if it is cooked. 
  • Pressing the centre of the cake to see if it springs back.
  • Hiring Paul Hollywood as a tester!

All of these checks are designed to see if the cake has been cooked satisfactorily. The shop decided to opt for all options, with the exception of hiring Paul Hollywood! Instead, each baker will review one another’s baking, with the hope that the business will grow to support a head baker who will review all cakes.

Just as the cake shop reviewed its processes to put a more stringent review structure in place, the same can also be implemented in a data-driven environment. The following questions are examples of some that you can ask yourself as part of this process:

  • Does the team need training?
  • Do you need to recruit new people with different skill sets?
  • Do the products need to change?
  • Is the supply line quick enough? 
  • Would more people or different processes help with efficiency?
  • Is the product still worth the effort that is invested in producing it?

Producing Gold

Once you’ve baked some quality cakes, you then need to take steps to market the product. It’s not enough to just produce the cakes and leave your customers to eat them if you want to maximise your marketing efforts. Building advocacy and influence via your customers is a great way of marketing your cakes to the right people. Word-of-mouth feedback from advocates is trusted by other potential customers, and is therefore far more effective than other forms of marketing.

However, there are two sides to the coin here. Negative feedback can be dangerous if it’s not managed effectively. Negative comments about the burnt edges of cakes will spread like wildfire to existing and prospective customers. But there are ways of recovering from this. Making courtesy calls to customers can provide you valuable insight into the process of ordering and consumption. 

Getting the right team in place, tailoring products for your target market, and taking feedback onboard in an active process are all important facets of contemporary marketing. 

What are the Considerations?

Gold Standards always begin with what you are looking to deliver and who this will benefit. Gold Standards should be designed to support outcomes, having considered both the internal and external factors that will influence design. Creating steps in the process to continually challenge the functionality of the end product and ensure that standards are still relevant to the end user should therefore be considered essential.

Sponsors and influencers can also play an extremely important role. Both can become prime advocates of your product, with the added benefit with sponsors that they pay you to advertise your goods or service!

Internal Considerations

Data

Data can be compared to ingredients within a cake. Naturally, good quality ingredients are critical to producing the best cake possible. The same applies to data. As we mentioned in previously, if you put rubbish data into your systems, you can expect rubbish outcomes!

At some point, the cake shop company realised that the bakers are not periodically reviewing the ingredients within their cupboards. To address this, the manager inspects everything that they have on hand, ensuring that any poor quality ingredients are replaced, and that anything out of data is thrown away. Labelling is updated, while processes are put in place to ensure that there is no repetition of these mistakes.

The key point here is that while complaints were registered about burnt edges, it may have been the ingredients that contributed to the final product that were the problem. Going forward, the team at the bakery put in place a series of key questions that would inform their processes in future baking:

  • Do we have enough data to make the size of cake required?
  • Are we getting our ingredients from the right suppliers? 
  • Does this product contain nuts?
  • Have I mixed sugar up with salt?
  • The milk smells as if it’s on the turn, should I use it?
  • Do we store the ingredients in the right place, in the correct containers? 

Resources and Teams

When baking your cake, you can select from many different types of bakers or specialist chefs to assist with the process. Or you may decide that you wish to train yourself, or an existing employee, so that they can handle the most challenging baking tasks.

In some cases, if the cake shop utilises industrial equipment, people who have been trained to use this equipment can be deployed, as opposed to bakers or specialist cake makers. Having the right team with the right skills, and/or the aptitude to learn them, can be critical to successfully achieving gold standards. Instilling this in your team culturally is critically important in providing direction to your whole operation.

Achieving this can be as simple as asking yourself the following questions:

  • Is the team right to build the end product? If not, what needs to change?
  • Is the team open to changing or evolving in order to improve the product or efficiency?
  • Do we have the right skills? If not, do we need to second or buy them in?

Company Culture

Finally, failing to understand the company culture will lead to failure. Gold standards must fit into the existing culture, or the direction the company is moving towards.

Understanding how your customers think, managing their expectations and developing a standard to consistently perform to those expectations is simple to conceive. However, the human element of this could result in you developing hundreds of different gold standards for multiple different customers. 

Important questions to ask yourself here:

  • Are the customers knowledgeable about your products? If not, can you educate them?
  • Do you share your practice? Would it help your customers to know what you do and how you do it?
  • Are there any expectations that you can manage? 
  • Are they any difficult expectations you have to work towards?

Implementing a gold standard for data may seem like an all-encompassing and intimidating goal. But it instead should be seen as a granular process. Breaking down the ingredients and individual components that collectively create gold standards is the best way to achieve this aim.


We have juxtaposed the process of dealing with data with the running of a cake shop. And when you’re developing the ideal processes for either type of business, simplicity is critically important.

But ‘process’ also refers to several aspects of the business. It can refer to the discussion of new orders with customers. It can be related to the process involved; the time it may take to bake and distribute cakes. It can also be related to the management of customer expectations and orders.

The process involved with the business can help your team understand what they do and how they fit in, as well as playing a vital role in deciphering and organising team responsibilities, interactions and achieving gold standards. It means that even if team members aren’t specialists in a particular area, they can still refer to established processes and be aware that they will result in a strong final product. For example, if a baker doesn’t know how to decorate a cake, they can understand that it can be decorated, and potentially the time that this will take. 

Understanding The Process

A good rule of thumb is to aim for the production team to understand enough about the process that they can make accurate estimates on production schedules. This demonstrates how much you can produce in one week if everyone works at 100% efficiency. And it also offers details of the effort needed to create the cake, and therefore the return or profit derived from the product. 

Once you have a good understanding of customer demand and production time, you can better manage cost and resourcing, potentially enabling you to offer alternatives to customers who have particular requirements. For example, customer demand may require the cake shop to produce 1,000 cakes in a week instead of just 100. By developing a process around the type of cake shop the establishment wishes to be in the local market, the orders can be successfully fulfilled.

In order to address this, the cake shop conducted some research on the market, reviewing the orders received over the last 12 months. Based on this data, they decided to optimise their operation so that they produced a consistent standard of high quality, mid-price range cakes using the best quality local produce. It is anticipated that this will constitute 80% of the company’s orders. Alongside this, some premium cakes will also be produced, with the intention of satisfying exclusive clientele, after the manager of the store noticed that this sort of cake tends to produce positive social media feedback.

With this in mind, the business decided to create a unified process, enabling her staff to process both types of orders using the same equipment. Different ingredients and alterations in the production process were required for the bakers to produce premium cakes, consequently more effort and attention to detail was needed. Furthermore, separate ingredient storage and labelling – to avoid premium ingredients being used within the standard cake – was also deemed essential.

Whether making cakes en masse to sell to local shops, or an artisan cake shop producing a range of bespoke, high quality and expensive cakes, each manufacturer will develop its own process to suit the desired output.

Questions you can ask yourself:

  • Do we understand what cake or range of cakes that we are trying to produce?
  • Can you easily explain your process to a customer, in terms they understand?
  • Is the manufacturing process suitable for the product/product range?
  • Do you understand the running costs involved with making more cakes?

Tools and Delivery

Producing gold standard cakes is not as simply as merely building a process or hiring the best people. In some cases, you can be limited by your tools. Realistically, there may be budgetary restraints that prevent you from purchasing the appropriate equipment for the job. So it can be best to utilise existing tools, but highlight them within the process as a potential risk to production. This will help with any customer discussions, particularly understanding any impact on production costs.

The new process at the cake shop enabled existing equipment, including the all-important ovens, to be fully utilised in the production of the desired range of cakes. As a result, business is soon booming and there is a large backlog of orders.

After assessing the production process and resources, the manager of the bakery decides that the most cost-effective way of increasing production is simply to add another oven within the production line. The additional oven enables the same staff and process to double the number of cakes that can be produced, while retaining the same processing time. The initial purchase price and ongoing maintenance costs are easily met by the increased revenue.

Questions you can ask yourself:

  • Do my tools enable me to produce our products effectively and efficiently?
  • What limitations do we have?
  • Do we utilise our tools correctly?
  • Do our production process utilise our tools in the best way possible?
  • Are there any other tools that can help me, either by saving resources or production time?

Time

Production time is often hidden from customers. All of your fantastic looking cakes in the shop window. or publicised on social media, take time to produce. Yet customers rarely appreciate the intricacy of the production process. Nor are they usually willing to wait! Therefore, managing expectations around deadlines from the very minute that any customer orders a cake will either address any concerns, which is why it’s always better to talk about this issue first.

Sometimes the thinking or desires of customers can change at the eleventh hour. On one occasion, the cake shop received a call from a customer late on Friday evening. At the last minute, Mr Jones had remembered that he needed to order a cake, so that it was ready for tomorrow’s big party. As one of the company’s bakers completed the order form, he realised that this order may not be possible, based on the existing orders that are already due to be baked tomorrow.  After a quick check with the kitchen staff, he confirms that the order cannot be satisfied, and subsequently informs Mr. Jones. The customer was disappointed, but understood after the kitchen capacity and pre-existing orders were explained to him.

Being transparent with customers and influencers about existing orders and production processes will help any business, but these processes will never stop unexpected orders from appearing at 8pm on a Friday night. However, it does create a consistent customer experience and help companies to mitigate any associated disappointment. Communication can be critical when all other approaches fall short.

Creating gold standards in data, or any business, is always dependent on efficient processes being put in place. These processes encompass every aspect of dealing with internal and external factors, and consequently it is critical for companies to continually refine every aspect of their operation.


Examining the importance of external considerations, these are factors inherent to the industry and location of your business that impact directly on your company and its day-to-day functioning.

Firstly, every industry has its own regulatory body and best practices that influence the way you work. In recent years, laws around the use of data (GDPR) and how it can be stored have evolved, with the aim of better protecting personal data.

To return to our cake shop example, the Food Safety Act 1990 provides the framework for all food legislation in England, Wales and Scotland. The legislation is concerned with, among other things, ensuring that food businesses take responsibility for how food is labelled, advertised and presented, so that it is neither false or misleading.

Laws and regulations therefore directly impact the processes, people and technology used to create cakes, while also potentially increasing production costs.

Questions you can ask yourself:

  • What regulatory bodies impact my cake shop, and how does this impact on the gold standard?
  • Are there any laws that may impact my cake production?
  • What if a new law or regulation arises, am I prepared for this? 

Economic (Demand drivers)

All customers have different demands; they require a range of products at varying price points. As we have discussed previously, the product price is dependent on the cost of production, however this is also influenced by the economic strength of the country in which the product is created.

Production Costs

The economic status of a country, whether recession or prosperity, has a direct impact on the cost of production and the purchasing habits of customers. Furthermore, various countries can produce the same products at different prices, while the strength of currencies can also impact the global market.

If a country is in recession, customers may shift from premium to mid-range products, accepting that the quality may diminish. Consumers may also purchase less frequently, or even stop buying entirely.

Here are some simple examples of economic impacts on the cake shop:

  • Recession may lead to using low-quality ingredients, or reducing the bakers that manufacture the products.
  • When the market is growing, this may lead to higher demand and / or production, or to an increased quality of ingredients.

The economy of a native country will drive the demand for cakes. This change in behaviour is something that cannot be controlled, but planning can be put in place. Building flexible and adaptable processes can enable your cake production to shift with demand, even though achieving this can be challenging. But identifying trends in demand is key to adapting efficiently and keeping your customers happy.

Questions you can ask yourself:

  • How can you detect changes in customer demand?
  • Are your gold standards set up for change? Can they adapt?
  • Are you and your team prepared for change?
  • Can your infrastructure cope with change?

Technology (Levers for improvement)

Technologies used in cake production are varied, but they must always support the processes and people involved in manufacturing.

Within the cake shop, the oven could be considered a bottleneck. If you have the space for more ovens then you can scale up your operation. And if there is an oven that can cook the same amount of cakes in half the time, this may be a better solution to the bottleneck.

Each item of technology within your process has its own limitations and potential. Understanding this will help you adapt to any situation that you encounter.

Questions you can ask yourself:

  • Is there a warranty on the ovens that we use?
  • Do I licence the equipment that we use, or do I own the equipment?
  • Do I understand the lifespan of our equipment?

Ongoing Delivery

The job can never be considered done, especially for data warehouses, particularly if data is used to make business decisions about customers, or to support business change.

Once the initial gold standard has been created, both periodic and failure reviews are recommended to ensure that standards continue to produce the value expected. These reviews can start at any point of the production process, but should be thorough and constructive, assessing all key elements of the people, processes and technology.

These reviews should be consistent in assessing elements against the following criteria:

  • Does the Process still fit – did anything about the process slow down the delivery?
  • What was the customer experience – did you create a good news story? Did the customer get what they wanted? Did this add value above what was expected?
  • Best Use of Technology – did the technology let you down?
  • Best use of People – were there any gaps in knowledge or skills, or impacts on customer experience?

Ask the difficult questions rather than reviewing the successes and you’ll have a much better chance of improving your operation.

Internal Considerations

As well as considering internal outcomes, there are also internal factors that will inevitably impact on the production process. Once production is up and running, there are usually support functions that are also part of the process, which are separate from the production team. These functions will help keep everything running smoothly. 

When the cake shop initially began trading, bakers were responsible for ensuring that the company’s machinery was in good working order and fit for purpose. This was a collective responsibility, but the bakers decided amongst themselves that one particular employee, Jerry, would carry out the maintenance, as he restored classic cars in his spare time and therefore already had an advanced skillset.

This was fine for some time. But when Jerry was out of the country on holiday, the food mixer broke down at the least opportune moment; right at the start of a large order. Jerry had the most experience and knowledge of this machine, with the other bakers having relied on him in these scenarios. In order to ensure that the order was fulfilled, the manager of the shop took the decision to purchase smaller food mixers, and pay the bakers overtime. The order was completed, and in the aftermath the management team reviewed the root cause of the delay in processing.

Subsequently, the decision was made to use an external maintenance company to monitor and maintain kitchen equipment. The shop’s equipment would be checked every month, to ensure that everything is in good working condition. In addition, the maintenance team will also be on call should something break during normal production hours.

The maintenance company ensures that all engineers have the relevant training and certifications to work on the machinery. Although the bakers may know enough to fix the machinery, it is not their job to deal with this, enabling them to focus solely on production.

But the shop continues to consult with the bakers when purchasing new equipment, as they keep abreast with the technology trends in the industry. This helps best practices be maintained.

Questions you can ask yourself:

  • Are roles and responsibilities suitable for your process?
  • Do you have any key risks?
  • Does your support team have the right experience?
  • Are there any service level agreements with the support team?
  • Does the support team add to the overall process?

Testing

Developing a testing method is another important internal consideration, yet it involves a simple process. Agile development is key here – revealing important data insight as it’s being developed. This constant checking builds certainty and comfort with business users, as they approve the state of data during development. This approach breaks down testing time and, in some cases, can reduce the overall development time. 

However, not all businesses can adopt this approach. Company policies, laws and regulatory requirements may impact the options available, and will be reflected within the internal process. This may lead to a more formal approach to testing.

Failure to control how testing is conducted can have a negative impact on the perception of the production team. Care is required to ensure that the team takes the right approach to obtain the correct output.

Questions you can ask yourself:

  • Does my testing methodology fit with the resource availability?
  • Is my testing thorough enough to meet business expectations?
  • Will the testing form part of my production process?

Creating gold standards requires internal and external considerations to be factored into the equation. Accounting for these diligently, and developing processes that reflect this, are critical for success in data-driven operations.


At Engaging Data, the implementation of the Gold Standard is something in which we always strive for and perpetuate in our work.

Deeply entwined in our work Engaging Data and The Gold Standard are synonymous.

Therefore, this being said, wanting to have a Data Consultancy in which you can have confidence in will be assured their work will be up to The Gold Standard, feel free to contact us for our services and products!


Gold Standards – Part 4

Gold Standards – Part 4

Engaging Data Explains :

Creating The Gold Standards in Data –

Part IV –

Balancing Internal and External Considerations


In this final part of our four-part blog on gold standards, we’re going to examine the importance of external considerations. These are factors inherent to the industry and location of your business that impact directly on your company and its day-to-day functioning.


Firstly, every industry has its own regulatory body and best practices that influence the way you work. In recent years, laws around the use of data (GDPR) and how it can be stored have evolved, with the aim of better protecting personal data.

To return to our cake shop example, the Food Safety Act 1990 provides the framework for all food legislation in England, Wales and Scotland. The legislation is concerned with, among other things, ensuring that food businesses take responsibility for how food is labelled, advertised and presented, so that it is neither false or misleading.

Laws and regulations therefore directly impact the processes, people and technology used to create cakes, while also potentially increasing production costs.

Questions you can ask yourself:

  • What regulatory bodies impact my cake shop, and how does this impact on the gold standard?
  • Are there any laws that may impact my cake production?
  • What if a new law or regulation arises, am I prepared for this? 

Economic (Demand drivers)

All customers have different demands; they require a range of products at varying price points. As we have discussed previously, the product price is dependent on the cost of production, however this is also influenced by the economic strength of the country in which the product is created.

Production Costs

The economic status of a country, whether recession or prosperity, has a direct impact on the cost of production and the purchasing habits of customers. Furthermore, various countries can produce the same products at different prices, while the strength of currencies can also impact the global market.

If a country is in recession, customers may shift from premium to mid-range products, accepting that the quality may diminish. Consumers may also purchase less frequently, or even stop buying entirely.

Here are some simple examples of economic impacts on the cake shop:

  • Recession may lead to using low-quality ingredients, or reducing the bakers that manufacture the products.
  • When the market is growing, this may lead to higher demand and / or production, or to an increased quality of ingredients.

The economy of a native country will drive the demand for cakes. This change in behaviour is something that cannot be controlled, but planning can be put in place. Building flexible and adaptable processes can enable your cake production to shift with demand, even though achieving this can be challenging. But identifying trends in demand is key to adapting efficiently and keeping your customers happy.

Questions you can ask yourself:

  • How can you detect changes in customer demand?
  • Are your gold standards set up for change? Can they adapt?
  • Are you and your team prepared for change?
  • Can your infrastructure cope with change?

Technology (Levers for improvement)

Technologies used in cake production are varied, but they must always support the processes and people involved in manufacturing.

Within the cake shop, the oven could be considered a bottleneck. If you have the space for more ovens then you can scale up your operation. And if there is an oven that can cook the same amount of cakes in half the time, this may be a better solution to the bottleneck.

Each item of technology within your process has its own limitations and potential. Understanding this will help you adapt to any situation that you encounter.

Questions you can ask yourself:

  • Is there a warranty on the ovens that we use?
  • Do I licence the equipment that we use, or do I own the equipment?
  • Do I understand the lifespan of our equipment?

Ongoing Delivery

The job can never be considered done, especially for data warehouses, particularly if data is used to make business decisions about customers, or to support business change.

Once the initial gold standard has been created, both periodic and failure reviews are recommended to ensure that standards continue to produce the value expected. These reviews can start at any point of the production process, but should be thorough and constructive, assessing all key elements of the people, processes and technology.

These reviews should be consistent in assessing elements against the following criteria:

  • Does the Process still fit – did anything about the process slow down the delivery?
  • What was the customer experience – did you create a good news story? Did the customer get what they wanted? Did this add value above what was expected?
  • Best Use of Technology – did the technology let you down?
  • Best use of People – were there any gaps in knowledge or skills, or impacts on customer experience?

Ask the difficult questions rather than reviewing the successes and you’ll have a much better chance of improving your operation.

Internal Considerations

As well as considering internal outcomes, there are also internal factors that will inevitably impact on the production process. Once production is up and running, there are usually support functions that are also part of the process, which are separate from the production team. These functions will help keep everything running smoothly. 

When the cake shop initially began trading, bakers were responsible for ensuring that the company’s machinery was in good working order and fit for purpose. This was a collective responsibility, but the bakers decided amongst themselves that one particular employee, Jerry, would carry out the maintenance, as he restored classic cars in his spare time and therefore already had an advanced skillset.

This was fine for some time. But when Jerry was out of the country on holiday, the food mixer broke down at the least opportune moment; right at the start of a large order. Jerry had the most experience and knowledge of this machine, with the other bakers having relied on him in these scenarios. In order to ensure that the order was fulfilled, the manager of the shop took the decision to purchase smaller food mixers, and pay the bakers overtime. The order was completed, and in the aftermath the management team reviewed the root cause of the delay in processing.

Subsequently, the decision was made to use an external maintenance company to monitor and maintain kitchen equipment. The shop’s equipment would be checked every month, to ensure that everything is in good working condition. In addition, the maintenance team will also be on call should something break during normal production hours.

The maintenance company ensures that all engineers have the relevant training and certifications to work on the machinery. Although the bakers may know enough to fix the machinery, it is not their job to deal with this, enabling them to focus solely on production.

But the shop continues to consult with the bakers when purchasing new equipment, as they keep abreast with the technology trends in the industry. This helps best practices be maintained.

Questions you can ask yourself:

  • Are roles and responsibilities suitable for your process?
  • Do you have any key risks?
  • Does your support team have the right experience?
  • Are there any service level agreements with the support team?
  • Does the support team add to the overall process?

Testing

Developing a testing method is another important internal consideration, yet it involves a simple process. Agile development is key here – revealing important data insight as it’s being developed. This constant checking builds certainty and comfort with business users, as they approve the state of data during development. This approach breaks down testing time and, in some cases, can reduce the overall development time. 

However, not all businesses can adopt this approach. Company policies, laws and regulatory requirements may impact the options available, and will be reflected within the internal process. This may lead to a more formal approach to testing.

Failure to control how testing is conducted can have a negative impact on the perception of the production team. Care is required to ensure that the team takes the right approach to obtain the correct output.

Questions you can ask yourself:

  • Does my testing methodology fit with the resource availability?
  • Is my testing thorough enough to meet business expectations?
  • Will the testing form part of my production process?

Creating gold standards requires internal and external considerations to be factored into the equation. Accounting for these diligently, and developing processes that reflect this, are critical for success in data-driven operations.