Microsoft Fabric is a cloud-based data analytics product that allows users to gain rich insights from vast amounts of data. These insights can include developing predictive models for data, visualizing data, and sharing data across the Microsoft Ecosystem. Microsoft Fabric makes all this possible. It’s a fully developed and integrated service that we offer as a Software as a Service (SaaS) product. Anyone in your company, including developers, data scientists, and business users, can use this tool.
Fabric itself consists of five applications that integrate seamlessly. This integration allows for quick and easy use between multiple apps. It also unifies data sources, as each application accesses all applications’ results and data, reducing the frequency of data copying or moving.The applications that make up Microsoft Fabric are OneLake, Data Factory, Synapse, Data Activator, and Power BI. All these apps share a common governance through Microsoft Purview which allows for easy control regarding auditing, security, privileges, and more. Also, all of these apps are or will be compatible with Microsoft CoPilot which will further help the user utilize AI to gain insights from data. Let us dive deeper into each application to see how they can help support your business!
OneLake – Like OneDrive, but for Databases
OneLake acts as a “lake” into which you can pour all your raw data into. A good analogy to help visualize this is to think of OneLake as the Pacific Ocean, the biggest ocean in the world. OneLake’s vast size unites all data. Admins organize these vast amounts of data by creating domains, also known as Lake Houses, where they group related data To visualize these domains/Lake houses, you can think of them as regions of the Pacific Ocean from our earlier analogy, like the Sea of Japan (East Sea) which is a small region of the huge Pacific Ocean.
But how do we import these vast amounts of data to start filling our OneLake? Users can use premade connectors to import data. We designed these connectors for compatibility between Microsoft Fabric and other platforms. If you have experience with the Power Platform, you can compare these connectors to those used for data import in apps like Power Automate and Power Apps. For OneLake, the connectors include importing from Dataverse, Azure Data Lake, and Amazon Web Services. Once imported, data can be managed by users in the OneLake Data Hub which is pictured below.
Data Factory – Connectors for Big and Small Datasets
Data Factory, another application in Microsoft Fabric, provides more ways to import data and ensures the data stays refreshed and up to date. It consists of Data Flows and Data Pipelines. Both tools allow users to import data into OneLake if the three connectors listed above are not adequate for your needs.
Data Flows are tools that help ingest and transform data to prepare it for analysis. These tools exist in a low-code interface to make it easier to understand and effectively use. These flows have access to 150+ connectors that allow for data ingestion from anywhere on the Internet. As mentioned above, these connectors are similar to those found in the Power Platform. Users are then able to transform this data using the Power Query Interface which is an interface that is available in existing Microsoft apps such as Excel. Transformations to data include joins, aggregations, cleansing, and more. Below is an example of what a data flow might look like.
The other tool included in Data Factory is Data Pipelines. Pipelines allow for cloud-scale capabilities to move petabyte-sized data. These tools also allow users to create automation that automatically refreshes their data to ensure that their numbers are always up to date. Overall pipelines can complete tasks at a high level of scale since they can use multiple dataflows at once with more complex code such as SQL scripts and Spark Notebooks in formats such as conditionals and loops.
Synapse – Modeling and Data Processing
Microsoft Synapse might sound familiar to those who have been working with data analytics in the Microsoft ecosystem. That is because Synapse has been around for longer than Fabric has. Synapse is a data analytics ecosystem that preceded Microsoft Fabric. Its official name is Azure Synapse Analytics and you can purchase it separately from Microsoft Fabric. However, without Fabric, it will not have seamless integration which makes data analytics more tedious depending on your data sources. Other than that Synapse behaves the same way in Microsoft Fabric including its four tools: Data Engineering, Data Warehouse, Data Science, and Real-Time Analytics. Because we are transferring these tools from a preexisting product, some of the Synapse tools serve almost the same exact purpose as other Fabric tools.
Data Engineering, the first tool in Microsoft Synapse, comes included in Microsoft Fabric. This tool enables users to create and manage data using Lakehouses, also known as domains Data Engineering also includes tools for Data ingestion, preparation, and transformation. Data Pipelines, a tool for data transformation, is part of Data Engineering, similar to Data Factory.
Data Warehouse, a OneLake-centric hub for data, is available as a Software-as-a-service tool. You can fully edit the data housed in this warehouse. The system automatically creates these warehouses for integration with other Synapse tools. However, you might be thinking, what is the difference between a Data Warehouse and OneLake? DataLakes in OneLake do not have a schema while Data Warehouses do. This means that data stored in Data warehouses are more structured in terms of SQL practices.
Data Science in Synapse is where analytics start. This tool allows users to begin creating Machine Learning models that can help predict future trends and patterns. This tool is most effective when the user has used Data Engineering and/or Data Factory to preprocess their data to prepare it for analysis. The term preprocessing means to transform raw data into a consistent format. For example, if you wanted to create a model to predict words, you would have to train the model on many different words and phrases.
However, these phrases have capital and lowercase letters as well as punctuation which could interfere with analysis. A form of preprocessing in this example would be to change all the words to be lowercase and remove all the punctuation. Once a user has done similar preprocessing with their data, they will be able to experiment with premade templated Machine Learning models, optimize their models to be most effective by tuning the parameters, and finally gain insights using their effective model. Creating your own predictive model without a template is also available.
Real-Time Analytics is the final tool from Microsoft Synapse included in Microsoft Fabric. If there’s one tool that’s most similar to Real-Time Analytics, it’s Power BI since both apps involve creating visual means to communicate information and insights. This app allows users to view seamless insights from the integrated Fabric platform. You get the most out of Real-time Analytics when you need insights to be fast, high-performance, and scalable for data analysis.
Data Activator – Trigger events based on data patterns
Data Activator, the fourth app in Microsoft Fabric, triggers events when it detects specific data patterns. It can do this because part of Data Activator’s software is to monitor data in real-time. Power BI also includes a feature called Events that accomplishes thisWith this feature, users can specify specific events that Data Activator should look out for.
When Data Activator detects these patterns, it can operate multiple services. One example of a service would be alerting the necessary people to a trend that could be critical. You can trigger these alerts and send them throughout the Microsoft 365 ecosystem, such as in Microsoft Teams or Microsoft Outlook. Another service Data Activator can run is Power Automate workflows. An example of this could be as follows: Data activator notices that Sales are declining for the third straight week. This pattern triggers a workflow in Power Automate that runs more ads on websites to help combat declining sales. Many more examples are possible, illustrating how useful Data Activator could be.
Power BI – Rich Visualizations for Your Data and Insights
Power BI is the last app we will be discussing that is included in Microsoft Fabric. This app, like Microsoft Synapse, already exists outside of Microsoft Fabric, meaning that you can purchase it on its own. However, purchasing it within Microsoft Fabric will guarantee seamless integration with the other apps discussed which will allow for a more user-friendly experience.
Power BI’s purpose is to create rich and pleasing graphics to represent your data. Examples of these graphics could include a map of the United States displaying the frequency of purchases in each state, a line graph showing the progression of sales throughout time, and many more. Through its integration into Microsoft Fabric, it will have access to all the data in your Fabric environment displaying it effectively with little to no code.
Governance in Microsoft Fabric
Microsoft Fabric also comes with built-in governance to make all your security implementations seamless and easy. Throughout the entire product, governance is taken care of through Microsoft Purview. Purview is a set of policies that offer solutions to protect and manage data. With data security in mind, Purview includes policies that preserve ownership by organizing data into domains that contain different permissions. This allows sensitive data to stay sensitive throughout your organization. Another policy that Purview includes in Fabric is Data Loss Prevention. This policy sends notifications when some sensitive data uploads or deletions are not approved. As you may have noticed, Purview is focused on the sensitivity of data which is why it has sensitivity labels for everything.
Other policies admins can expect are policies that aid in auditing. For example, some policies log and store user activities in Fabric. You can use this data to demonstrate met requirements and assist in forensic investigations. Purview collects this information automatically, and it’s sensitive, and viewable only by admin accounts.
The pricing for Microsoft Fabric is different from that of other Microsoft apps. With Fabric, pricing is dependent on the amount of computing power you need as well as the amount of storage. But first, what is computing power? Computing power refers to the amount of processing power your projects will need so that the data analysis does not take too long. For example, it will take less computing power to process thousands of pieces of data as compared to millions of billions of pieces of data. The more data you have, the more storage and computing power you will need.
In terms of computing power, Fabric is priced as a pay-as-you-go method. The method includes pricing for both monthly and hourly rates depending on the use case. The lowest charge for computing is the “F 2” option which is $0.36/hour or $262.80/month. From here the computing power increases by powers of two with the highest computing option being “F 2048” which is priced at $368.64/hour or $269,107.20/month. All computing power in Microsoft is cloud-based, so you can think of this as paying to use X amount of computing power for Y amount of time in a Microsoft data center. These options can also be purchased in reservation so that you can access the computing power when you need it.
In terms of storage, Fabric is priced similarly with customers being charged per Gigabyte. Currently, the OneLake store is $0.023/GB/month. Other options such as OneLake BCDR storage and OneLake Cache are priced differently and offer additional features to users using OneLake.
In conclusion, Microsoft Fabric emerges as a powerful and seamlessly integrated solution for data analytics. It offers a suite of applications designed to meet the diverse needs of users across various roles in your organization. From OneLake’s expansive data unification to Data Factory’s versatile data import capabilities, Synapse’s robust modeling and processing, Data Activator’s event-triggered actions, and Power BI’s rich visualizations, Fabric provides a comprehensive platform. With built-in governance through Microsoft Purview, your data remains secure and well-managed. As you embark on leveraging the capabilities of Microsoft Fabric for your business, Total Solutions Inc. stands ready to assist you. For any inquiries, questions, or personalized guidance on implementing Microsoft Fabric, don’t hesitate to reach out to us. Elevate your data analytics experience with Total Solutions Inc. – your trusted partner in navigating the vast possibilities of Microsoft Fabric.
Source – Data Analytics | Microsoft Fabric