Innovations, Insights, and Future Directions
FabCon Europe 2024, Microsoft’s first Fabric Community Conference, has proven to be a pivotal event in the world of data and technology. This year’s conference brought together more than 3,300 industry experts, leaders, and innovators from 61 countries around the globe to share the latest advancements, discuss challenges, and explore future trends.
Product and Broader Theme Takeaways
Effective data governance is crucial – As well as extensive discussions on how Fabric can meet governance needs in the era of AI, there was detailed coverage of both Fabric’s built-in governance features and utilising Fabric alongside Purview for extended data governance and security. An important quote highlighted that: “Without proper governance and security, analytics initiatives are at risk of producing unreliable or compromised results.”
Fabric as an AI-embedded experience – The discussions explored this from multiple angles, including Copilot-driven development and consumption, generative AI solutions, integrating OneLake data into custom-built AI applications, and the ability to call AI functions (e.g., classify, translate) directly via notebooks. Key points covered included how generative AI is not always the right solution and the importance of governing data to support AI effectively.
Power Hour was a thoroughly enjoyable experience and is highly recommended for those unfamiliar with it. The energy on stage, combined with the lighthearted and engaging nature of the demos, was a masterclass in storytelling during presentations and demonstrated how to have fun with data. It reiterated a point that most passionate data practitioners understand: the value of data is often determined by the narrative built around it, and how it can be explained in simple, understandable terms to the business.
A focus on ease of use – Microsoft emphasised minimising the barriers to entry with the extension of Copilot features (in PowerBI and building custom Copilots on Fabric data), the addition of Fabric semantic model authoring via desktop, UI changes, the ability to embed real-time dashboards, and features for ease of implementation/migration. This included integrating Azure Data Factory pipelines directly into Fabric, sessions on migrating from Azure Synapse, and upcoming support (in 2024) for migrating existing SSIS jobs.
Community Takeaways
The community aspect – Before attending, it was unclear what the branding of a “community conference” would entail. However, the conference made a dedicated effort to engage the community, from the “Fast with Fabric” developer challenge focused on understanding how people are using the tool, to continuously seeking product feedback for future development. The community booth remained busy throughout the event.
The Fabric community is massive and growing rapidly. With 3,300 attendees from 61 countries, significant Microsoft representation, and 14,000+ Fabric customers, the activity in the Fabric space is extensive. One observation was that Fabric is on a similar trajectory as PowerBI at the same point in its lifecycle—considering the nearly 400,000 PowerBI customers today, there is clearly a large target for growth.
User growth observations – Two key insights emerged: Fabric has the fastest-growing certification Microsoft offers (Analytics Engineer certification, DP-600), with over 17,000 certified individuals. Additionally, many users come from non-technical or non-data backgrounds, only turning to data solutions when addressing specific problems (often using PowerBI). It’s refreshing to see many now considering data alongside business problems rather than as an afterthought.
Engaging with the community is incredibly valuable, especially in person. Whether it was resolving specific technical queries, validating design decisions and best practices, or simply learning about what others are working on and how. There was much to gain from speaking with Microsoft staff, MVPs at the “Ask the Experts” booths, session leaders, and other attendees.
Best Practice Takeaways
Optimising your semantic models is crucial – While DirectLake can improve value and performance, a poorly optimized model outside Fabric will remain suboptimal within it.
V-Order for read operations – V-Order is primarily intended for read operations, so careful consideration is needed for its usage. The best practice suggested was to use V-Order for “gold” layer data feeding into semantic models and to use the Delta analyzer to examine specific details.
DirectLake and query folding – Using DirectLake is as critical in Fabric as query folding is in PowerBI. DirectLake can significantly enhance read performance; an example cited was reading 2 billion rows in 13 seconds in a P1 capacity via direct query and 1 billion rows in 234ms via DirectLake. While the record counts differ, this was intentional to avoid a 2 billion record read that would force a direct query fallback.
Spark optimisation – Spark optimisation has many aspects, but one key area discussed was spark session startup times. Two recommended strategies were using starter pools and high concurrency mode. High concurrency mode enables multiple sessions to share Spark resources, reducing startup times to as little as 5 seconds.
Conclusion
FabCon Europe 2024 was a treasure trove of insights, innovations, and community engagement. The event underscored the importance of effective data governance, the transformative potential of AI, and the need for a user-centric approach to product development. As we move forward, the lessons and connections forged at FabCon will undoubtedly drive collective progress in the realms of data and technology.
