Why Microsoft Fabric Is Becoming the New Operating System for Enterprise Data

1
00:00:00,000 –> 00:00:03,120
You don’t have a data platform, you have a staged illusion,

2
00:00:03,120 –> 00:00:06,960
power BI pretending to be glue, pipelines masking drift,

3
00:00:06,960 –> 00:00:09,440
access you can’t explain, lineage you can’t prove,

4
00:00:09,440 –> 00:00:11,920
you call it analytics, it’s chaos.

5
00:00:11,920 –> 00:00:15,440
Your truth lives in 15 places, non-authoritative,

6
00:00:15,440 –> 00:00:17,120
you blame slow BI.

7
00:00:17,120 –> 00:00:19,760
It’s not BI, it’s entropy.

8
00:00:19,760 –> 00:00:23,200
The system you built multiplies copies, hides logic,

9
00:00:23,200 –> 00:00:24,800
and leaks credentials.

10
00:00:24,800 –> 00:00:27,040
Here’s the thing you weren’t supposed to hear,

11
00:00:27,040 –> 00:00:31,600
one system from raw to inside, fabric, one lake,

12
00:00:31,600 –> 00:00:34,640
medallion, one access path.

13
00:00:34,640 –> 00:00:37,040
But there’s a bug that ruins everything.

14
00:00:37,040 –> 00:00:39,600
You keep smuggling business logic into bronze.

15
00:00:39,600 –> 00:00:42,480
If you can’t name your access path and your contracts,

16
00:00:42,480 –> 00:00:44,160
you don’t have a platform.

17
00:00:44,160 –> 00:00:46,960
You have sprawl why fabric exists,

18
00:00:46,960 –> 00:00:49,040
the hidden cost of fragmentation,

19
00:00:49,040 –> 00:00:51,200
you want the real reason fabric exists,

20
00:00:51,200 –> 00:00:54,480
not the promo, the autopsy, fragmentation,

21
00:00:54,480 –> 00:00:57,040
everywhere, fragmented tools, fragmented logic,

22
00:00:57,040 –> 00:00:58,960
fragmented security, fragmented monitoring,

23
00:00:58,960 –> 00:01:00,800
each team solves their local pain.

24
00:01:00,800 –> 00:01:03,520
Nobody owns the whole, the result isn’t innovation,

25
00:01:03,520 –> 00:01:04,800
it’s copy storms.

26
00:01:04,800 –> 00:01:06,560
Tell me if this sounds like you,

27
00:01:06,560 –> 00:01:08,240
marketing spins up a warehouse,

28
00:01:08,240 –> 00:01:09,760
finance builds a mart,

29
00:01:09,760 –> 00:01:12,640
ups, dumps, CSVs in a lake.

30
00:01:12,640 –> 00:01:16,320
BI stitches the mess with just one more semantic model.

31
00:01:16,320 –> 00:01:19,520
Now there are seven truths, none trusted.

32
00:01:19,520 –> 00:01:21,360
This is your first failure mode,

33
00:01:21,360 –> 00:01:26,240
silo proliferation, copies breed, costs climb,

34
00:01:26,240 –> 00:01:30,240
trust drops, then schema drift,

35
00:01:30,240 –> 00:01:33,280
columns mutate without warning.

36
00:01:33,280 –> 00:01:37,040
Status becomes state, nulls gain meaning.

37
00:01:37,040 –> 00:01:39,280
Your pipelines pretend it’s fine,

38
00:01:39,280 –> 00:01:42,400
until a CEO metric dies quietly on a Sunday night.

39
00:01:42,400 –> 00:01:43,760
You call it a refresh issue.

40
00:01:43,760 –> 00:01:46,160
It’s not, it’s governance rot,

41
00:01:46,160 –> 00:01:48,320
security split brain next,

42
00:01:48,320 –> 00:01:50,640
service principles in five tools,

43
00:01:50,640 –> 00:01:52,560
personal tokens buried in YAML,

44
00:01:52,560 –> 00:01:55,920
a secret in a notebook you forgot to rotate.

45
00:01:55,920 –> 00:01:58,240
Every platform has its own ACLs,

46
00:01:58,240 –> 00:01:59,840
its own audit story,

47
00:01:59,840 –> 00:02:01,760
its own MFA edge cases.

48
00:02:01,760 –> 00:02:05,040
No single sentence answers who can see what,

49
00:02:05,040 –> 00:02:06,720
so you invent theater.

50
00:02:06,720 –> 00:02:09,760
A SharePoint spreadsheet called Access Matrix.

51
00:02:09,760 –> 00:02:11,040
Nobody trusts it either.

52
00:02:11,040 –> 00:02:13,120
And then the bills, in just engine here,

53
00:02:13,120 –> 00:02:15,600
transform engine there, storage in three places,

54
00:02:15,600 –> 00:02:17,120
BI premium on top,

55
00:02:17,120 –> 00:02:20,560
egress fees leaking value on the margins,

56
00:02:20,560 –> 00:02:24,000
you store the same truth four times in slightly different shapes.

57
00:02:24,000 –> 00:02:26,880
You overpay for duplication and latency.

58
00:02:26,880 –> 00:02:28,720
Underneath the simple root cause,

59
00:02:28,720 –> 00:02:31,200
fragmentation fabric is an attack on that root cause,

60
00:02:31,200 –> 00:02:33,280
not a new chart, not a rebrand,

61
00:02:33,280 –> 00:02:35,200
a surface area kill shot.

62
00:02:35,200 –> 00:02:36,560
One identity,

63
00:02:36,560 –> 00:02:37,520
Entra,

64
00:02:37,520 –> 00:02:40,560
the same identity that hits your inbox is the identity

65
00:02:40,560 –> 00:02:41,920
that hits your lake.

66
00:02:41,920 –> 00:02:43,920
Conditional Access MFA groups,

67
00:02:43,920 –> 00:02:46,240
one story, no shadow token forms,

68
00:02:46,240 –> 00:02:48,320
one storage layer, one lake,

69
00:02:48,320 –> 00:02:50,880
open tables, delta format,

70
00:02:50,880 –> 00:02:53,040
shortcuts instead of copies,

71
00:02:53,040 –> 00:02:55,440
you don’t shuttle files between clouds,

72
00:02:55,440 –> 00:02:56,800
you reference them.

73
00:02:56,800 –> 00:02:58,400
The contract is the table,

74
00:02:58,400 –> 00:02:59,360
not the pipeline.

75
00:02:59,360 –> 00:03:02,240
The storage is the platform, not the afterthought.

76
00:03:02,240 –> 00:03:03,920
One governance plane,

77
00:03:03,920 –> 00:03:05,760
Per view plus workspaces,

78
00:03:05,760 –> 00:03:08,320
lineage sensitivity data map roles,

79
00:03:08,320 –> 00:03:10,080
same console, same audit,

80
00:03:10,080 –> 00:03:13,840
Access rides with items inside a workspace boundary,

81
00:03:13,840 –> 00:03:16,480
not duct taped across 10 portals,

82
00:03:16,480 –> 00:03:19,760
discoverability is built in, not bolted on.

83
00:03:19,760 –> 00:03:20,880
One monitoring view,

84
00:03:20,880 –> 00:03:22,640
Capacities workloads, pipelines,

85
00:03:22,640 –> 00:03:24,480
notebooks, semantic models,

86
00:03:24,480 –> 00:03:26,320
one place to see pressure,

87
00:03:26,320 –> 00:03:27,520
one place to prove health,

88
00:03:27,520 –> 00:03:29,040
one place to hunt failure,

89
00:03:29,040 –> 00:03:30,800
that’s the unification bet,

90
00:03:30,800 –> 00:03:33,520
shared identity, shared storage,

91
00:03:33,520 –> 00:03:35,040
shared governance,

92
00:03:35,040 –> 00:03:36,480
shared monitoring,

93
00:03:36,480 –> 00:03:38,080
reduced surface area,

94
00:03:38,080 –> 00:03:39,920
reduced hiding spots.

95
00:03:39,920 –> 00:03:42,000
Now the part you don’t want to admit,

96
00:03:42,000 –> 00:03:44,400
the cost of ignoring this isn’t just money.

97
00:03:44,400 –> 00:03:45,680
It’s cognitive load,

98
00:03:45,680 –> 00:03:48,560
every engineer in your shop keeps a map in their head.

99
00:03:48,560 –> 00:03:50,320
Which tool transforms this table?

100
00:03:50,320 –> 00:03:51,440
Where does the raw land?

101
00:03:51,440 –> 00:03:53,120
What refresh triggers what?

102
00:03:53,120 –> 00:03:55,600
Who’s allowed to fix it when it breaks?

103
00:03:55,600 –> 00:03:57,520
Each inconsistency adds friction.

104
00:03:57,520 –> 00:03:59,040
Each friction adds delay.

105
00:03:59,040 –> 00:04:00,320
Delay becomes a habit,

106
00:04:00,320 –> 00:04:01,680
habits become culture,

107
00:04:01,680 –> 00:04:03,280
culture becomes drift.

108
00:04:03,280 –> 00:04:05,280
And your users, they see the show,

109
00:04:05,280 –> 00:04:06,320
the dashboard loads,

110
00:04:06,320 –> 00:04:07,520
the numbers field off,

111
00:04:07,520 –> 00:04:08,800
they export to excel,

112
00:04:08,800 –> 00:04:10,000
shadow truth blooms,

113
00:04:10,000 –> 00:04:12,160
the platform you build to unify reality

114
00:04:12,160 –> 00:04:14,160
becomes a confusion factory.

115
00:04:14,160 –> 00:04:15,840
Fabric is a reset,

116
00:04:15,840 –> 00:04:16,880
not magic,

117
00:04:16,880 –> 00:04:18,240
just opinionated.

118
00:04:18,240 –> 00:04:19,680
One lake instead of five,

119
00:04:19,680 –> 00:04:22,560
one security posture instead of five contradictory ones,

120
00:04:22,560 –> 00:04:26,000
one capacity meter instead of a cluster of random invoices,

121
00:04:26,000 –> 00:04:28,400
one place to name your access path,

122
00:04:28,400 –> 00:04:30,080
define your contracts,

123
00:04:30,080 –> 00:04:31,920
and force discipline.

124
00:04:31,920 –> 00:04:33,280
Bronze raw.

125
00:04:33,280 –> 00:04:35,200
Silver validated, gold modeled,

126
00:04:35,200 –> 00:04:36,880
logic never smuggled backward.

127
00:04:36,880 –> 00:04:39,520
Let’s be clear about the medallion’s stake in the ground

128
00:04:39,520 –> 00:04:42,560
because this is where most shops lie to themselves.

129
00:04:42,560 –> 00:04:44,240
Bronze is ingestion only.

130
00:04:44,240 –> 00:04:46,160
No business rules,

131
00:04:46,160 –> 00:04:47,280
no deduping,

132
00:04:47,280 –> 00:04:48,400
no joins.

133
00:04:48,400 –> 00:04:51,440
It’s an immutable log of what arrived when from where?

134
00:04:51,440 –> 00:04:53,680
Silver is where you enforce contracts,

135
00:04:53,680 –> 00:04:54,880
types,

136
00:04:54,880 –> 00:04:56,080
nullability,

137
00:04:56,080 –> 00:04:57,680
deduplication,

138
00:04:57,680 –> 00:04:59,360
conformance,

139
00:04:59,360 –> 00:05:02,080
slowly changing dimensions if you need them.

140
00:05:02,080 –> 00:05:03,920
Documented semantics,

141
00:05:03,920 –> 00:05:05,760
tested transforms,

142
00:05:05,760 –> 00:05:08,800
gold is consumption-specific models,

143
00:05:08,800 –> 00:05:10,400
curated metrics,

144
00:05:10,400 –> 00:05:12,000
star schemas,

145
00:05:12,000 –> 00:05:14,800
business logic exposed on purpose,

146
00:05:14,800 –> 00:05:16,560
not hidden in a notebook.

147
00:05:16,560 –> 00:05:18,000
You violate that boundary,

148
00:05:18,000 –> 00:05:19,280
you invite decay,

149
00:05:19,280 –> 00:05:21,120
you bury a quick fix in bronze,

150
00:05:21,120 –> 00:05:22,560
you destroy lineage,

151
00:05:22,560 –> 00:05:24,720
you shove a one-off join into a dashboard,

152
00:05:24,720 –> 00:05:26,160
you destroy reuse,

153
00:05:26,160 –> 00:05:28,080
bad logic thrives in the gaps,

154
00:05:28,080 –> 00:05:29,920
fragmentation feeds on ambiguity.

155
00:05:29,920 –> 00:05:32,000
So fabric compresses those gaps,

156
00:05:32,000 –> 00:05:34,320
workspaces become your security boundary.

157
00:05:34,320 –> 00:05:38,000
One lake becomes your single storage plane.

158
00:05:38,000 –> 00:05:41,520
Per view becomes your discovery and lineage map,

159
00:05:41,520 –> 00:05:45,760
direct lake makes your semantic model the shortest path from gold to inside.

160
00:05:45,760 –> 00:05:47,200
You still have complexity,

161
00:05:47,200 –> 00:05:48,320
but it’s contained,

162
00:05:48,320 –> 00:05:49,840
you still have freedom,

163
00:05:49,840 –> 00:05:50,960
but it’s fenced.

164
00:05:50,960 –> 00:05:53,200
You don’t have to love Microsoft to see the math.

165
00:05:53,200 –> 00:05:56,160
If your life cycle is fragmented, governance is fiction.

166
00:05:56,160 –> 00:05:57,680
If governance is fiction,

167
00:05:57,680 –> 00:05:59,840
trusted analytics are theater.

168
00:05:59,840 –> 00:06:01,600
Unify the life cycle or drown.

169
00:06:01,600 –> 00:06:02,480
That’s the offer,

170
00:06:02,480 –> 00:06:04,160
that’s why fabric exists.

171
00:06:04,160 –> 00:06:05,440
Not to add features,

172
00:06:05,440 –> 00:06:06,800
to remove excuses,

173
00:06:06,800 –> 00:06:08,480
what fabric actually is,

174
00:06:08,480 –> 00:06:09,920
the one platform moment.

175
00:06:09,920 –> 00:06:10,480
Look,

176
00:06:10,480 –> 00:06:12,080
you don’t need another logo.

177
00:06:12,080 –> 00:06:14,160
You need fewer places to hide failure.

178
00:06:14,160 –> 00:06:15,680
Fabric isn’t a feature bundle.

179
00:06:15,680 –> 00:06:16,960
It’s a reduction.

180
00:06:16,960 –> 00:06:20,240
It folds the life cycle into one access path

181
00:06:20,240 –> 00:06:22,640
and dares you to stop lying to yourself.

182
00:06:22,640 –> 00:06:23,920
Start with the spine,

183
00:06:23,920 –> 00:06:24,560
one lake.

184
00:06:24,560 –> 00:06:26,640
It’s your organization’s data lake,

185
00:06:26,640 –> 00:06:27,360
singular,

186
00:06:27,360 –> 00:06:30,000
backed by open delta-parkay tables,

187
00:06:30,000 –> 00:06:31,200
not compatible,

188
00:06:31,200 –> 00:06:32,160
native tables,

189
00:06:32,160 –> 00:06:33,760
not files are the contract.

190
00:06:33,760 –> 00:06:37,200
Shortcuts reference external stores without copying.

191
00:06:37,200 –> 00:06:38,320
Your S3 bucket,

192
00:06:38,320 –> 00:06:40,720
referenced, your ADLS container,

193
00:06:40,720 –> 00:06:41,760
referenced,

194
00:06:41,760 –> 00:06:43,680
no more shuttle scripts that rot.

195
00:06:43,680 –> 00:06:44,800
The lake is the platform,

196
00:06:44,800 –> 00:06:46,080
not a staging accident.

197
00:06:46,080 –> 00:06:48,240
On top of that fabric mounts experiences,

198
00:06:48,240 –> 00:06:50,400
not separate products with separate security,

199
00:06:50,400 –> 00:06:52,400
views on the same storage plane.

200
00:06:52,400 –> 00:06:55,200
Data factory for ingest and orchestration,

201
00:06:55,200 –> 00:06:56,960
data engineering for spark notebooks

202
00:06:56,960 –> 00:06:58,560
and lake house management,

203
00:06:58,560 –> 00:07:01,040
data science for experiments and models,

204
00:07:01,040 –> 00:07:04,240
warehouse for t-school over delta with the Polaris engine,

205
00:07:04,240 –> 00:07:07,840
real-time analytics for KQL over streaming,

206
00:07:07,840 –> 00:07:10,560
power BI for semantic models and reports,

207
00:07:10,560 –> 00:07:13,840
data activator for reflexes when data crosses a line,

208
00:07:13,840 –> 00:07:14,800
different doors,

209
00:07:14,800 –> 00:07:15,840
same floor,

210
00:07:15,840 –> 00:07:16,800
identity,

211
00:07:16,800 –> 00:07:17,360
entry,

212
00:07:17,360 –> 00:07:18,320
full stop,

213
00:07:18,320 –> 00:07:19,520
conditional access,

214
00:07:19,520 –> 00:07:20,320
MFA,

215
00:07:20,320 –> 00:07:21,120
groups,

216
00:07:21,120 –> 00:07:23,040
workspaces as the security boundary,

217
00:07:23,040 –> 00:07:25,200
you assign people to a workspace role.

218
00:07:25,200 –> 00:07:27,440
Those roles govern every item inside,

219
00:07:27,440 –> 00:07:29,600
lake houses, warehouses, pipelines,

220
00:07:29,600 –> 00:07:31,680
notebooks, semantic models, reports,

221
00:07:31,680 –> 00:07:33,360
no parallel ACLs patrol,

222
00:07:33,360 –> 00:07:36,160
no personal tokens hiding in a markdown cell.

223
00:07:36,720 –> 00:07:39,200
One sentence now answers who can see what?

224
00:07:39,200 –> 00:07:40,320
Governance?

225
00:07:40,320 –> 00:07:43,200
Per view rides alongside automatic lineage

226
00:07:43,200 –> 00:07:45,520
across items in the workspace,

227
00:07:45,520 –> 00:07:46,560
scan the lake house,

228
00:07:46,560 –> 00:07:48,160
see downstream semantic models,

229
00:07:48,160 –> 00:07:49,280
see reports,

230
00:07:49,280 –> 00:07:51,680
apply sensitivity labels once they travel.

231
00:07:51,680 –> 00:07:53,280
Discovery from the data map,

232
00:07:53,280 –> 00:07:56,320
glossary terms that actually bind to assets,

233
00:07:56,320 –> 00:07:58,480
so you don’t invent lineage slides anymore.

234
00:07:58,480 –> 00:07:59,600
You show the graph,

235
00:07:59,600 –> 00:08:01,840
monitoring the capacities the heartbeat,

236
00:08:01,840 –> 00:08:03,200
one meter for compute,

237
00:08:03,200 –> 00:08:06,400
one monitoring hub to see pipeline failures,

238
00:08:06,400 –> 00:08:07,760
notebook runs,

239
00:08:07,760 –> 00:08:09,360
warehouse queries,

240
00:08:09,360 –> 00:08:11,280
direct lake refreshes,

241
00:08:11,280 –> 00:08:12,640
report activity,

242
00:08:12,640 –> 00:08:13,840
heat by workload,

243
00:08:13,840 –> 00:08:15,360
spikes by time,

244
00:08:15,360 –> 00:08:17,120
correlate events across the stack

245
00:08:17,120 –> 00:08:20,240
without all tabbing through five admin portals.

246
00:08:20,240 –> 00:08:21,440
When something fails,

247
00:08:21,440 –> 00:08:22,480
you see the chain.

248
00:08:22,480 –> 00:08:25,280
Now the medallion architecture becomes enforceable.

249
00:08:25,280 –> 00:08:26,480
Not aspirational.

250
00:08:26,480 –> 00:08:27,760
Bronze is the landing zone

251
00:08:27,760 –> 00:08:29,360
in the lake house files area,

252
00:08:29,360 –> 00:08:30,320
immutable,

253
00:08:30,320 –> 00:08:32,960
append only source granular,

254
00:08:32,960 –> 00:08:34,560
schema drift captured,

255
00:08:34,560 –> 00:08:35,360
not fixed.

256
00:08:36,160 –> 00:08:38,720
Silver is delta tables in the tables area

257
00:08:38,720 –> 00:08:40,560
that enforce types to dupes,

258
00:08:40,560 –> 00:08:42,800
surrogate keys, conformance,

259
00:08:42,800 –> 00:08:45,200
declarative transforms in notebooks

260
00:08:45,200 –> 00:08:46,240
or data flows,

261
00:08:46,240 –> 00:08:47,040
gen2,

262
00:08:47,040 –> 00:08:48,080
tested,

263
00:08:48,080 –> 00:08:49,120
versioned,

264
00:08:49,120 –> 00:08:50,480
documented,

265
00:08:50,480 –> 00:08:52,080
gold is curated,

266
00:08:52,080 –> 00:08:54,000
dimensional models,

267
00:08:54,000 –> 00:08:55,680
fact and dimension,

268
00:08:55,680 –> 00:08:56,960
delta tables,

269
00:08:56,960 –> 00:08:58,320
surface to power BI

270
00:08:58,320 –> 00:09:00,480
as direct lake semantic models,

271
00:09:00,480 –> 00:09:01,520
measures live in DAX,

272
00:09:01,520 –> 00:09:02,800
not buried upstream,

273
00:09:02,800 –> 00:09:04,560
that boundaries your discipline.

274
00:09:04,560 –> 00:09:06,160
Fabric gives you the rails,

275
00:09:06,160 –> 00:09:07,840
you decide if you stay on them.

276
00:09:07,840 –> 00:09:10,000
Direct lake changes the BI equation.

277
00:09:10,000 –> 00:09:13,600
The semantic model reads delta files directly from one lake.

278
00:09:13,600 –> 00:09:14,480
No import bloat,

279
00:09:14,480 –> 00:09:16,240
no direct query latency tags,

280
00:09:16,240 –> 00:09:17,440
vertipack pages,

281
00:09:17,440 –> 00:09:19,440
data into memory on demand.

282
00:09:19,440 –> 00:09:20,640
Update the goal table,

283
00:09:20,640 –> 00:09:22,880
the model reframes to new files.

284
00:09:22,880 –> 00:09:24,640
You control auto detect

285
00:09:24,640 –> 00:09:26,880
versus scheduled reframing,

286
00:09:26,880 –> 00:09:28,960
freshness without nightly fragility,

287
00:09:28,960 –> 00:09:30,720
performance without copy storms.

288
00:09:30,720 –> 00:09:32,800
Orchestration,

289
00:09:32,800 –> 00:09:35,680
pipeline centralized triggers dependencies,

290
00:09:35,680 –> 00:09:36,800
retries,

291
00:09:36,800 –> 00:09:38,480
triggers spark notebooks,

292
00:09:38,480 –> 00:09:40,640
kick a warehouse stored procedure,

293
00:09:40,640 –> 00:09:42,000
refreshes semantic model,

294
00:09:42,000 –> 00:09:43,440
one schedule spine,

295
00:09:43,440 –> 00:09:45,120
one failure surface,

296
00:09:45,120 –> 00:09:46,480
one audit trail,

297
00:09:46,480 –> 00:09:48,400
the what runs when answer

298
00:09:48,400 –> 00:09:50,400
stops living in someone’s head.

299
00:09:50,400 –> 00:09:52,240
Real time isn’t bolted on.

300
00:09:52,240 –> 00:09:54,880
Event streams shape ingestion from Kafka,

301
00:09:54,880 –> 00:09:57,040
event hubs, MQTT,

302
00:09:57,040 –> 00:09:59,120
route to KQL databases

303
00:09:59,120 –> 00:10:01,200
for sub-second analytics,

304
00:10:01,200 –> 00:10:04,720
land snapshots to one lake for medallion promotion.

305
00:10:04,720 –> 00:10:07,840
Your streaming and batch aren’t estranged cousins.

306
00:10:07,840 –> 00:10:09,680
There are two lanes on the same road.

307
00:10:09,680 –> 00:10:11,440
KQL for fast questions,

308
00:10:11,440 –> 00:10:12,960
delta for durable truth,

309
00:10:12,960 –> 00:10:14,800
AI is not marketing confetti,

310
00:10:14,800 –> 00:10:16,240
copilot drafts pipelines,

311
00:10:16,240 –> 00:10:17,760
SQL, notebook code,

312
00:10:17,760 –> 00:10:19,040
DAX measures.

313
00:10:19,040 –> 00:10:20,560
It reads your data map

314
00:10:20,560 –> 00:10:22,400
to suggest joins and keys.

315
00:10:22,400 –> 00:10:24,800
It’s not a substitute for architecture.

316
00:10:24,800 –> 00:10:26,880
It’s a speed boost for the boring parts.

317
00:10:26,880 –> 00:10:28,400
The guard rail still matter.

318
00:10:28,400 –> 00:10:30,000
Bronze stays raw,

319
00:10:30,000 –> 00:10:31,920
silver stays validated,

320
00:10:31,920 –> 00:10:33,680
gold stays modeled,

321
00:10:33,680 –> 00:10:36,240
copilot accelerates within the fence,

322
00:10:36,240 –> 00:10:37,280
licensing,

323
00:10:37,280 –> 00:10:38,800
one fabric capacity,

324
00:10:38,800 –> 00:10:40,960
F-SQ in Azure,

325
00:10:40,960 –> 00:10:43,360
UMETA workloads not products,

326
00:10:43,360 –> 00:10:46,480
a trusted workspace mode for network controls,

327
00:10:46,480 –> 00:10:48,240
cost management at the subscription.

328
00:10:48,240 –> 00:10:50,160
You stop guessing which SKU

329
00:10:50,160 –> 00:10:51,840
hit the egress bill this month,

330
00:10:51,840 –> 00:10:53,120
one invoice,

331
00:10:53,120 –> 00:10:54,560
one throttle,

332
00:10:54,560 –> 00:10:55,680
predictable.

333
00:10:55,680 –> 00:10:57,680
Security posture tightens

334
00:10:57,680 –> 00:10:59,920
because the surface area shrinks,

335
00:10:59,920 –> 00:11:02,880
the data never leaves one lake unless you say so.

336
00:11:02,880 –> 00:11:04,080
Shortcuts point in,

337
00:11:04,080 –> 00:11:05,920
they don’t spray copies out.

338
00:11:05,920 –> 00:11:09,120
Row level and object level security

339
00:11:09,120 –> 00:11:12,480
live at the semantic model for consumption,

340
00:11:12,480 –> 00:11:13,520
workspace,

341
00:11:13,520 –> 00:11:15,520
rolls for platform access,

342
00:11:15,520 –> 00:11:17,920
classified data carries labels into reports.

343
00:11:17,920 –> 00:11:19,200
Audits land in one place.

344
00:11:19,200 –> 00:11:21,760
You don’t beg five teams for five logs.

345
00:11:21,760 –> 00:11:23,280
This is the one platform moment.

346
00:11:23,280 –> 00:11:24,320
Storage unified,

347
00:11:24,320 –> 00:11:25,360
identity unified,

348
00:11:25,360 –> 00:11:26,400
governance unified,

349
00:11:26,400 –> 00:11:27,440
monitoring unified,

350
00:11:27,440 –> 00:11:28,240
cost unified.

351
00:11:28,240 –> 00:11:29,440
The medallion layers

352
00:11:29,440 –> 00:11:31,920
stop being slides and become folders,

353
00:11:31,920 –> 00:11:34,160
tables and models you can point to.

354
00:11:34,160 –> 00:11:36,400
The semantic model stops being a bandage

355
00:11:36,400 –> 00:11:38,480
and becomes the official interface.

356
00:11:38,480 –> 00:11:40,400
The BI team stops being the glue

357
00:11:40,400 –> 00:11:42,080
and becomes a consumer of gold,

358
00:11:42,080 –> 00:11:43,600
not a maker of silver.

359
00:11:43,600 –> 00:11:45,280
You still have to do the hard part,

360
00:11:45,280 –> 00:11:46,560
name your access path,

361
00:11:46,560 –> 00:11:47,920
define your contracts,

362
00:11:47,920 –> 00:11:49,280
lock your boundaries,

363
00:11:49,280 –> 00:11:51,680
but now the system enforces your intent

364
00:11:51,680 –> 00:11:53,040
instead of undermining it.

365
00:11:53,040 –> 00:11:55,200
Fewer tools, fewer tokens,

366
00:11:55,200 –> 00:11:57,520
fewer copies, fewer excuses.

367
00:11:57,520 –> 00:11:59,520
You don’t need another hero feature.

368
00:11:59,520 –> 00:12:00,880
You need one place to stand.

369
00:12:00,880 –> 00:12:02,240
Fabric gives you that ground.

370
00:12:02,240 –> 00:12:04,080
Now either use it

371
00:12:04,080 –> 00:12:06,080
or keep exporting to excel in the dark.

372
00:12:06,080 –> 00:12:08,400
The medallion architecture,

373
00:12:08,400 –> 00:12:10,160
from available to usable,

374
00:12:10,160 –> 00:12:11,440
you don’t start with truth,

375
00:12:11,440 –> 00:12:12,560
you start with evidence,

376
00:12:12,560 –> 00:12:13,440
then you prove it,

377
00:12:13,440 –> 00:12:14,320
then you present it.

378
00:12:14,320 –> 00:12:15,200
That’s medallion,

379
00:12:15,200 –> 00:12:16,720
bronze, silver, gold,

380
00:12:16,720 –> 00:12:17,520
not a vibe,

381
00:12:17,520 –> 00:12:18,160
a contract.

382
00:12:18,160 –> 00:12:20,400
Bronze is not a playground.

383
00:12:20,400 –> 00:12:22,160
It’s a black box flight recorder,

384
00:12:22,160 –> 00:12:22,880
immutable,

385
00:12:22,880 –> 00:12:24,080
append only,

386
00:12:24,080 –> 00:12:25,360
source granular,

387
00:12:25,360 –> 00:12:26,960
you land exactly what arrived

388
00:12:26,960 –> 00:12:27,760
when it arrived,

389
00:12:27,760 –> 00:12:28,560
where it came from,

390
00:12:28,560 –> 00:12:29,520
no dedoops,

391
00:12:29,520 –> 00:12:30,320
no joins,

392
00:12:30,320 –> 00:12:31,280
no quick fixes.

393
00:12:31,280 –> 00:12:33,440
If the source shipped nulls

394
00:12:33,440 –> 00:12:34,560
in a required field,

395
00:12:34,560 –> 00:12:35,440
that’s the point.

396
00:12:35,440 –> 00:12:37,040
You keep the fracture visible,

397
00:12:37,040 –> 00:12:38,720
because the moment you clean bronze,

398
00:12:38,720 –> 00:12:40,160
you erase the crime scene.

399
00:12:40,160 –> 00:12:40,800
In fabric,

400
00:12:40,800 –> 00:12:42,400
that’s your lake house files area,

401
00:12:42,400 –> 00:12:44,080
one folder per source system,

402
00:12:44,080 –> 00:12:46,720
one partitioning strategy per ingestion cadence.

403
00:12:46,720 –> 00:12:48,480
By day, by hour,

404
00:12:48,480 –> 00:12:50,240
by slice, that matches arrival.

405
00:12:50,240 –> 00:12:52,160
Delta lockstract changes,

406
00:12:52,160 –> 00:12:53,840
you capture schema drift,

407
00:12:53,840 –> 00:12:54,880
don’t conceal it.

408
00:12:54,880 –> 00:12:56,320
You store raw JSON,

409
00:12:56,320 –> 00:12:58,720
as is, and a typed landing if needed,

410
00:12:58,720 –> 00:13:00,080
but both are still bronze.

411
00:13:00,080 –> 00:13:02,800
Your only transforms are structural,

412
00:13:02,800 –> 00:13:04,960
normalized line breaks,

413
00:13:04,960 –> 00:13:07,040
split oversized files,

414
00:13:07,040 –> 00:13:08,960
stamp metadata columns,

415
00:13:08,960 –> 00:13:09,840
arrivals,

416
00:13:09,840 –> 00:13:11,120
the source system,

417
00:13:11,120 –> 00:13:12,640
correlation ID,

418
00:13:12,640 –> 00:13:14,160
evidence preserved.

419
00:13:14,160 –> 00:13:15,520
Now the question moves from

420
00:13:15,520 –> 00:13:16,640
what arrived,

421
00:13:16,640 –> 00:13:17,760
to what’s valid,

422
00:13:17,760 –> 00:13:18,480
that’s silver.

423
00:13:18,480 –> 00:13:22,320
This is the layer where you enforce expectations.

424
00:13:22,320 –> 00:13:23,120
Types,

425
00:13:23,120 –> 00:13:24,240
ranges,

426
00:13:24,240 –> 00:13:26,000
referential integrity,

427
00:13:26,000 –> 00:13:28,480
did duplication with deterministic keys,

428
00:13:28,480 –> 00:13:30,880
late arriving logic formalized,

429
00:13:30,880 –> 00:13:31,840
not improvised,

430
00:13:31,840 –> 00:13:33,440
here you pin meaning to columns.

431
00:13:33,440 –> 00:13:36,480
Status code is not free text,

432
00:13:36,480 –> 00:13:39,120
it references a dimension table you govern,

433
00:13:39,120 –> 00:13:40,640
you declare nullability,

434
00:13:40,640 –> 00:13:42,720
you document semantics next to code,

435
00:13:42,720 –> 00:13:44,560
and you test it,

436
00:13:44,560 –> 00:13:45,280
row counts,

437
00:13:45,280 –> 00:13:46,320
uniqueness,

438
00:13:46,320 –> 00:13:47,680
referential checks,

439
00:13:47,680 –> 00:13:48,960
drift alerts.

440
00:13:48,960 –> 00:13:50,640
Silver is your usable truth,

441
00:13:50,640 –> 00:13:51,840
not yet curated,

442
00:13:51,840 –> 00:13:53,680
solid enough to reuse.

443
00:13:53,680 –> 00:13:57,040
In fabric, silver lives in lake house tables as delta.

444
00:13:57,040 –> 00:14:00,000
It’s built with notebooks or data flows gen 2.

445
00:14:00,000 –> 00:14:02,000
The technique is simple and strict,

446
00:14:02,000 –> 00:14:04,800
read bronze with schema on read.

447
00:14:04,800 –> 00:14:06,800
Profile, reject bad rows

448
00:14:06,800 –> 00:14:09,200
into a quarantine table with reasons,

449
00:14:09,200 –> 00:14:11,600
conform types apply SCD rules

450
00:14:11,600 –> 00:14:14,640
if the business demands slowly changing state,

451
00:14:14,640 –> 00:14:16,800
did duplicate by composite key plus

452
00:14:16,800 –> 00:14:18,160
a winning record rule,

453
00:14:18,160 –> 00:14:20,000
latest event by event keys,

454
00:14:20,000 –> 00:14:20,720
for example,

455
00:14:20,720 –> 00:14:22,720
maintain a hash of business columns

456
00:14:22,720 –> 00:14:24,480
for change detection,

457
00:14:24,480 –> 00:14:25,920
stamp lineage columns,

458
00:14:25,920 –> 00:14:27,520
bronze path,

459
00:14:27,520 –> 00:14:29,600
silver version.

460
00:14:29,600 –> 00:14:31,840
Every transform is idempotent,

461
00:14:31,840 –> 00:14:32,960
every write is atomic,

462
00:14:32,960 –> 00:14:34,640
here’s where most teams cheat,

463
00:14:34,640 –> 00:14:36,800
they slip business logic into silver,

464
00:14:36,800 –> 00:14:38,080
just this once,

465
00:14:38,080 –> 00:14:40,800
they add a special join for a stubborn KPI.

466
00:14:40,800 –> 00:14:41,440
Don’t,

467
00:14:41,440 –> 00:14:42,880
if it encodes business meaning,

468
00:14:42,880 –> 00:14:43,920
it belongs in gold,

469
00:14:43,920 –> 00:14:45,760
silver encodes reality,

470
00:14:45,760 –> 00:14:47,040
gold encodes purpose,

471
00:14:47,040 –> 00:14:49,680
gold is consumption specific,

472
00:14:49,680 –> 00:14:52,000
it’s the model your users touch.

473
00:14:52,000 –> 00:14:54,080
Curated dimensional structures,

474
00:14:54,080 –> 00:14:55,600
conformed dimensions,

475
00:14:55,600 –> 00:14:57,920
fact tables with consistent grain,

476
00:14:57,920 –> 00:15:01,040
build to answer stable questions fast.

477
00:15:01,040 –> 00:15:02,880
This is where you lift measures,

478
00:15:02,880 –> 00:15:04,160
revenue, churn,

479
00:15:04,160 –> 00:15:05,200
ARR,

480
00:15:05,200 –> 00:15:08,320
into DAX with definitions you can defend.

481
00:15:08,320 –> 00:15:11,360
This is where row level security lives for consumption.

482
00:15:11,360 –> 00:15:13,200
This is where you shape experiences,

483
00:15:13,200 –> 00:15:14,560
not data.

484
00:15:14,560 –> 00:15:16,800
In fabric gold is still delta in one lake,

485
00:15:16,800 –> 00:15:19,680
but now you expose it through semantic models,

486
00:15:19,680 –> 00:15:20,880
indirect lake.

487
00:15:20,880 –> 00:15:22,640
Fact tables are thin,

488
00:15:22,640 –> 00:15:24,880
wide where it helps compression,

489
00:15:24,880 –> 00:15:28,160
with integer surrogate keys to dimensions,

490
00:15:28,160 –> 00:15:31,120
dimensions carry histories when they matter.

491
00:15:31,120 –> 00:15:34,240
The star schema stops being an academic exercise

492
00:15:34,240 –> 00:15:37,200
and becomes a performance and governance tool.

493
00:15:37,200 –> 00:15:39,200
Direct leg reads the files directly,

494
00:15:39,200 –> 00:15:40,240
no import lag,

495
00:15:40,240 –> 00:15:41,920
no direct query tax.

496
00:15:41,920 –> 00:15:43,440
If you keep your tables tidy,

497
00:15:43,440 –> 00:15:45,520
partitions align to refresh windows,

498
00:15:45,520 –> 00:15:46,800
small dimension tables,

499
00:15:46,800 –> 00:15:47,920
fully in memory,

500
00:15:47,920 –> 00:15:49,120
measure sane,

501
00:15:49,120 –> 00:15:50,640
you get speed without copies.

502
00:15:50,640 –> 00:15:52,720
The contract between layers is your guardrail,

503
00:15:52,720 –> 00:15:54,560
so put it in words and code.

504
00:15:54,560 –> 00:15:56,720
Bronze contract, immutable append,

505
00:15:56,720 –> 00:15:58,240
no business transforms,

506
00:15:58,240 –> 00:16:01,040
source fidelity retained, drift captured.

507
00:16:01,040 –> 00:16:03,520
Silver contract, validated schema,

508
00:16:03,520 –> 00:16:04,800
documented semantics,

509
00:16:04,800 –> 00:16:07,680
deduped conforming joins only for reference data,

510
00:16:07,680 –> 00:16:10,560
test coverage that blocks publish on failure.

511
00:16:10,560 –> 00:16:13,120
Gold contract, curated stars,

512
00:16:13,120 –> 00:16:14,720
certified measures,

513
00:16:14,720 –> 00:16:17,600
RLS rules documented and tested,

514
00:16:17,600 –> 00:16:20,960
semantic model source bound to specific tables,

515
00:16:20,960 –> 00:16:23,040
no ad hoc sources.

516
00:16:23,040 –> 00:16:24,880
Enforced those contracts with gates,

517
00:16:24,880 –> 00:16:27,600
pipelines won’t advance if silver tests fail.

518
00:16:27,600 –> 00:16:31,200
Semantic models don’t reframe if gold partitions aren’t complete.

519
00:16:31,200 –> 00:16:33,760
Perview tags apply at silver and flow

520
00:16:33,760 –> 00:16:36,400
through to gold reports automatically.

521
00:16:36,400 –> 00:16:38,800
Workspace Rolesgate, who can write to which layer,

522
00:16:38,800 –> 00:16:41,120
you design the friction on purpose.

523
00:16:41,120 –> 00:16:44,000
Edge cases, streaming, don’t panic.

524
00:16:44,000 –> 00:16:46,240
Bronze handles late and out of order events

525
00:16:46,240 –> 00:16:47,760
by watermark and window.

526
00:16:47,760 –> 00:16:51,040
Silver computes a canonical state table

527
00:16:51,040 –> 00:16:54,240
and an events table at different grains.

528
00:16:54,240 –> 00:16:55,840
Gold exposes both,

529
00:16:55,840 –> 00:16:57,440
state for current dashboards,

530
00:16:57,440 –> 00:16:59,520
events for investigations.

531
00:16:59,520 –> 00:17:01,440
Edge case two, third party marts,

532
00:17:01,440 –> 00:17:02,720
you can’t change,

533
00:17:02,720 –> 00:17:04,320
shortcut them into bronze,

534
00:17:04,320 –> 00:17:06,080
mirror their schema into silver

535
00:17:06,080 –> 00:17:07,840
with a thin validation layer.

536
00:17:07,840 –> 00:17:09,360
Only then join in gold,

537
00:17:09,360 –> 00:17:11,920
keep their chaos outside your contracts.

538
00:17:11,920 –> 00:17:14,560
If you remember nothing else, remember this.

539
00:17:14,560 –> 00:17:16,160
Bronze answers what happened.

540
00:17:16,160 –> 00:17:18,880
Silver answers what is true.

541
00:17:18,880 –> 00:17:20,320
Gold answers what it means.

542
00:17:20,320 –> 00:17:22,640
Mix them and you lose the plot.

543
00:17:22,640 –> 00:17:25,520
Keep them clean and the rest of fabric clicks.

544
00:17:25,520 –> 00:17:28,400
Multi workspace architecture,

545
00:17:28,400 –> 00:17:29,920
the Nord1 approach,

546
00:17:29,920 –> 00:17:31,600
you don’t secure chaos.

547
00:17:31,600 –> 00:17:33,200
You contain it, that’s not one.

548
00:17:33,200 –> 00:17:34,400
A climb with fixed anchors,

549
00:17:34,400 –> 00:17:35,680
not vibes, boundaries.

550
00:17:35,680 –> 00:17:39,200
We use multiple workspaces with purpose.

551
00:17:39,200 –> 00:17:41,520
Each workspace is a boundary for security,

552
00:17:41,520 –> 00:17:43,120
cost, change control.

553
00:17:43,120 –> 00:17:45,520
Roads are consistent, names are boring.

554
00:17:45,520 –> 00:17:46,160
That’s the point.

555
00:17:46,160 –> 00:17:49,280
At the top, a platform workspace set.

556
00:17:49,280 –> 00:17:51,840
Three tiers, dev, test,

557
00:17:51,840 –> 00:17:53,840
prod, inside each,

558
00:17:53,840 –> 00:17:55,920
core lake house, warehouse,

559
00:17:55,920 –> 00:17:57,440
shared dimensions,

560
00:17:57,440 –> 00:17:58,960
data factory pipelines,

561
00:17:58,960 –> 00:18:00,080
monitoring artifacts.

562
00:18:00,080 –> 00:18:01,680
This is platform, not project.

563
00:18:01,680 –> 00:18:03,920
It’s where medallion lives as a service.

564
00:18:03,920 –> 00:18:06,240
Then domain workspaces,

565
00:18:06,240 –> 00:18:08,480
finance, sales,

566
00:18:08,480 –> 00:18:11,200
ops, HR, supply chain.

567
00:18:11,840 –> 00:18:14,240
Again, dev, test,

568
00:18:14,240 –> 00:18:16,160
prod for each domain.

569
00:18:16,160 –> 00:18:18,560
Domains don’t own silver infrastructure.

570
00:18:18,560 –> 00:18:21,920
They own gold models and domain specific transforms.

571
00:18:21,920 –> 00:18:24,640
They depend on the platform silver via shortcuts.

572
00:18:24,640 –> 00:18:26,880
Separation of concerns,

573
00:18:26,880 –> 00:18:28,800
no one sneaks logic upstream.

574
00:18:28,800 –> 00:18:31,520
One more tier, shared analytics,

575
00:18:31,520 –> 00:18:33,120
certified semantic models,

576
00:18:33,120 –> 00:18:34,960
conformed metrics,

577
00:18:34,960 –> 00:18:36,880
cross-domain star schemas.

578
00:18:36,880 –> 00:18:39,360
This is the only place with the official measures

579
00:18:39,360 –> 00:18:41,120
the business can certify.

580
00:18:41,120 –> 00:18:42,720
Everything else is a draft.

581
00:18:42,720 –> 00:18:43,840
Walk it with me.

582
00:18:43,840 –> 00:18:45,120
Hybrid narration.

583
00:18:45,120 –> 00:18:46,720
UI callouts.

584
00:18:46,720 –> 00:18:49,040
In fabric, open the left rail.

585
00:18:49,040 –> 00:18:50,080
Click workspaces.

586
00:18:50,080 –> 00:18:51,120
You’ll see folders.

587
00:18:51,120 –> 00:18:52,800
No, but you’ll name like folders.

588
00:18:52,800 –> 00:18:54,080
Prefix everything.

589
00:18:54,080 –> 00:18:55,920
NW, plat, dev,

590
00:18:55,920 –> 00:18:57,920
NW, plat, tst,

591
00:18:57,920 –> 00:18:59,760
NW, plat, prd,

592
00:18:59,760 –> 00:19:01,520
NW, fin, dev.

593
00:19:01,520 –> 00:19:03,200
Same pattern for sales,

594
00:19:03,200 –> 00:19:04,640
ops, HR, supply.

595
00:19:04,640 –> 00:19:06,720
The NW prefix is your org tag.

596
00:19:06,720 –> 00:19:08,000
The middle is the domain.

597
00:19:08,000 –> 00:19:09,840
The suffix is the environment.

598
00:19:09,840 –> 00:19:11,440
Consistency kills doubt.

599
00:19:11,440 –> 00:19:15,120
Open NW, plat, dev, items,

600
00:19:15,120 –> 00:19:17,840
Lakehouse, NW, plat, dev, core, lake,

601
00:19:17,840 –> 00:19:19,920
warehouse, NW, plat, dev, core,

602
00:19:19,920 –> 00:19:22,560
WH, pipelines, NW, plat, dev,

603
00:19:22,560 –> 00:19:24,160
ingest, NW, plat, dev,

604
00:19:24,160 –> 00:19:25,840
promote bronze to silver,

605
00:19:25,840 –> 00:19:28,880
NW, plat, dev, promote silver to gold.

606
00:19:28,880 –> 00:19:31,920
Notebooks, validation suites, monitoring,

607
00:19:31,920 –> 00:19:34,720
a report pinned from monitoring hub with capacity

608
00:19:34,720 –> 00:19:35,920
and pipeline views,

609
00:19:35,920 –> 00:19:37,360
filtered to this workspace.

610
00:19:37,360 –> 00:19:38,320
This is your spine.

611
00:19:38,320 –> 00:19:39,120
Security.

612
00:19:39,120 –> 00:19:41,840
Workspace roles, viewers, contributors, members,

613
00:19:41,840 –> 00:19:44,240
admins, platform team, only as admins.

614
00:19:44,240 –> 00:19:46,160
Data engineers is members for dev,

615
00:19:46,160 –> 00:19:49,920
but readers in test, prod, unless change window opens.

616
00:19:49,920 –> 00:19:52,400
Business users, they don’t enter platform.

617
00:19:52,400 –> 00:19:56,240
Ever.StorageAlignment, one lake, data hub,

618
00:19:56,240 –> 00:20:00,640
in core lake, files, area holds, bronze per source folders,

619
00:20:00,640 –> 00:20:04,720
tables, area holds, silver, and conform dimensions.

620
00:20:04,720 –> 00:20:07,600
Under settings, turn on trusted workspace access

621
00:20:07,600 –> 00:20:10,000
and network isolation as required.

622
00:20:10,000 –> 00:20:11,680
You want predictable blast radius.

623
00:20:11,680 –> 00:20:17,280
Now domains, open NW sales dev items,

624
00:20:17,280 –> 00:20:21,040
lake house, NW sales dev, LH,

625
00:20:21,040 –> 00:20:26,320
semantic models, NW sales dev, gold sales ops,

626
00:20:26,320 –> 00:20:28,400
reports drafts only.

627
00:20:28,400 –> 00:20:29,520
Here’s the trick.

628
00:20:29,520 –> 00:20:30,960
In the domain lake house,

629
00:20:30,960 –> 00:20:34,000
create shortcuts to platform silver tables.

630
00:20:34,000 –> 00:20:36,240
Sales orders, customers, products,

631
00:20:36,240 –> 00:20:40,720
do not copy. Shortcuts point to the platform lake house tables path.

632
00:20:40,720 –> 00:20:43,200
You inherit data lineage security posture.

633
00:20:43,200 –> 00:20:47,040
Your domain adds its own transforms for gold only.

634
00:20:47,040 –> 00:20:50,080
Segmentation tables, derived facts,

635
00:20:50,080 –> 00:20:52,080
domain specific dimensions.

636
00:20:52,080 –> 00:20:54,480
Per view, go to Microsoft Per view hub,

637
00:20:54,480 –> 00:20:58,160
register the ND-plat PRD lake house and warehouse assources,

638
00:20:58,160 –> 00:21:00,960
scan on a schedule, turn on lineage.

639
00:21:00,960 –> 00:21:04,480
Now open lineage on a certified sales semantic model

640
00:21:04,480 –> 00:21:06,160
in shared analytics.

641
00:21:06,160 –> 00:21:08,800
You’ll see the path, shared model,

642
00:21:08,800 –> 00:21:11,760
domain gold tables, shortcut backed,

643
00:21:11,760 –> 00:21:15,120
platform silver, platform bronze source connectors.

644
00:21:15,120 –> 00:21:16,400
You didn’t draw that diagram.

645
00:21:16,400 –> 00:21:17,360
The system did.

646
00:21:17,360 –> 00:21:18,960
That’s how you sleep.

647
00:21:18,960 –> 00:21:21,360
Monitoring, open monitoring hub,

648
00:21:21,360 –> 00:21:24,640
filter by workspace NW-plat PRD.

649
00:21:24,640 –> 00:21:28,480
Pipelines tab, in just success rate, duration cost,

650
00:21:28,480 –> 00:21:31,920
notebooks tab, validation notebook failures,

651
00:21:31,920 –> 00:21:35,680
warehouse tab, query heat for Polaris.

652
00:21:35,680 –> 00:21:39,840
Power BI tab, reframing latency for direct lake models.

653
00:21:39,840 –> 00:21:42,880
One screen, one heartbeat, cost,

654
00:21:42,880 –> 00:21:46,080
fabric capacity in Azure, assign heavy workspaces

655
00:21:46,080 –> 00:21:47,520
to dedicated capacities.

656
00:21:47,520 –> 00:21:50,800
NW-plat PRD on F64 or above if you needed,

657
00:21:50,800 –> 00:21:52,960
domains share in F32 perhaps,

658
00:21:52,960 –> 00:21:54,480
label capacities with tags,

659
00:21:54,480 –> 00:21:56,800
ENV-it prod, domain-to-platform.

660
00:21:56,800 –> 00:22:00,400
Use Azure cost management budgets and alerts.

661
00:22:00,400 –> 00:22:03,760
No guessing, no mystery invoice.

662
00:22:04,400 –> 00:22:06,000
Promotion flow.

663
00:22:06,000 –> 00:22:10,960
Platform first, bronze lands via data factory in NW-plat dev.

664
00:22:10,960 –> 00:22:12,720
Validation notebooks run.

665
00:22:12,720 –> 00:22:17,440
If tests pass a pipeline promotes code via Git branches to test,

666
00:22:17,440 –> 00:22:18,720
data mirrors a subset.

667
00:22:18,720 –> 00:22:20,640
If test passes same to prod,

668
00:22:20,640 –> 00:22:23,280
domains consume via shortcuts.

669
00:22:23,280 –> 00:22:25,760
Their gold models move through dev,

670
00:22:25,760 –> 00:22:28,240
test, prod, with deployment pipelines,

671
00:22:28,240 –> 00:22:31,680
parameterized connections to the corresponding platform workspace.

672
00:22:31,680 –> 00:22:34,160
No hard-coded paths.

673
00:22:34,160 –> 00:22:36,240
Only environment parameters.

674
00:22:36,240 –> 00:22:38,320
Access path clarity.

675
00:22:38,320 –> 00:22:41,200
Business consumers never touch domain workspaces.

676
00:22:41,200 –> 00:22:43,280
They hit shared analytics,

677
00:22:43,280 –> 00:22:45,200
certified models only.

678
00:22:45,200 –> 00:22:47,440
RLS maps to ENTRA groups.

679
00:22:47,440 –> 00:22:49,600
Group names align with domains.

680
00:22:49,600 –> 00:22:53,440
NRLS sales, EMIA, NRLS sales, NAY.

681
00:22:53,440 –> 00:22:54,800
The platform owns the pattern.

682
00:22:54,800 –> 00:22:56,480
Domains populate the membership,

683
00:22:56,480 –> 00:22:57,600
common mistakes.

684
00:22:57,600 –> 00:22:59,920
Domains writing to platform silver.

685
00:22:59,920 –> 00:23:03,280
No, block contributor writes in platform for domain teams.

686
00:23:03,280 –> 00:23:06,480
Domains copying platform tables to their own storage.

687
00:23:06,480 –> 00:23:09,040
No, enforce shortcuts.

688
00:23:09,040 –> 00:23:11,840
Platform sneaking KPIs into silver.

689
00:23:11,840 –> 00:23:15,520
No, code review, gate, with a rule.

690
00:23:15,520 –> 00:23:19,520
Any computed business measure belongs in gold or semantic DAX.

691
00:23:19,520 –> 00:23:21,760
Why not want works?

692
00:23:21,760 –> 00:23:25,120
Fixed anchors, clear lanes.

693
00:23:25,120 –> 00:23:26,960
Platform is the slope.

694
00:23:26,960 –> 00:23:28,240
Domains are routes.

695
00:23:28,240 –> 00:23:30,720
Shared analytics is the summit photo.

696
00:23:30,720 –> 00:23:32,960
You contain complexity to known surfaces.

697
00:23:32,960 –> 00:23:34,800
You reduce places to lie.

698
00:23:34,800 –> 00:23:36,080
You make failure loud.

699
00:23:36,080 –> 00:23:38,720
And when someone asks, “Where does this number come from?”

700
00:23:38,720 –> 00:23:39,920
You click lineage.

701
00:23:39,920 –> 00:23:40,960
You don’t improvise.

702
00:23:40,960 –> 00:23:42,160
You show the access path.

703
00:23:42,160 –> 00:23:44,640
Enterprise proof.

704
00:23:44,640 –> 00:23:46,800
Three scenarios that expose the edge.

705
00:23:46,800 –> 00:23:47,920
You want proof.

706
00:23:47,920 –> 00:23:48,960
Not slides.

707
00:23:48,960 –> 00:23:52,000
Three operations where fragmentation kills you.

708
00:23:52,000 –> 00:23:54,560
And where fabrics single access path holds.

709
00:23:54,560 –> 00:23:55,760
Real time ops first.

710
00:23:55,760 –> 00:23:57,600
The place outages hide.

711
00:23:57,600 –> 00:23:59,440
Ingest is an event stream.

712
00:23:59,440 –> 00:24:01,200
Not a nightly CSV.

713
00:24:01,200 –> 00:24:03,600
Devices published a limit to event hubs.

714
00:24:03,600 –> 00:24:06,160
In fabric, event streams listens.

715
00:24:06,160 –> 00:24:07,520
We define a route.

716
00:24:07,520 –> 00:24:08,720
Raw telemetry.

717
00:24:08,720 –> 00:24:11,200
KQL database for hot analytics.

718
00:24:11,200 –> 00:24:13,360
One-Lake bronze for durability.

719
00:24:13,360 –> 00:24:14,560
No copies.

720
00:24:14,560 –> 00:24:17,520
Shortcuts expose bronze to the platform lake house.

721
00:24:17,520 –> 00:24:20,240
Silver builds two tables.

722
00:24:20,240 –> 00:24:24,080
Events at raw grain and device stayed by last known window

723
00:24:24,080 –> 00:24:25,360
with a watermark.

724
00:24:25,360 –> 00:24:30,080
We validate ranges, discard impossible values to quarantine

725
00:24:30,080 –> 00:24:31,360
and stamp lineage.

726
00:24:31,360 –> 00:24:33,680
Gold shapes two paths.

727
00:24:33,680 –> 00:24:36,800
For operators, a direct lake semantic model over device

728
00:24:36,800 –> 00:24:38,880
and state and a small incidence fact.

729
00:24:38,880 –> 00:24:39,520
Measures.

730
00:24:39,520 –> 00:24:40,320
MTTR.

731
00:24:40,320 –> 00:24:41,200
Active alerts.

732
00:24:41,200 –> 00:24:42,640
SLA breaches.

733
00:24:42,640 –> 00:24:43,680
For investigators.

734
00:24:43,680 –> 00:24:46,640
A KQL dashboard for sub-second queries on events.

735
00:24:46,640 –> 00:24:48,880
Data activator locks the loop.

736
00:24:48,880 –> 00:24:49,680
Condition.

737
00:24:49,680 –> 00:24:50,480
Temperature.

738
00:24:50,480 –> 00:24:52,080
Threshold for N minutes.

739
00:24:52,080 –> 00:24:54,400
And device and state not acknowledged.

740
00:24:54,400 –> 00:24:55,680
Action.

741
00:24:55,680 –> 00:24:56,720
Post to teams.

742
00:24:56,720 –> 00:24:57,920
Create a ticket.

743
00:24:57,920 –> 00:24:59,680
Write an incident row.

744
00:24:59,680 –> 00:25:01,200
One workspace owns the flow.

745
00:25:01,200 –> 00:25:02,480
One lake holds the truth.

746
00:25:02,480 –> 00:25:04,800
KQL and Delta share the spine.

747
00:25:04,800 –> 00:25:05,600
No mirrors.

748
00:25:05,600 –> 00:25:07,200
No hidden refreshes.

749
00:25:07,200 –> 00:25:10,400
And when an exec asks why an alert fired.

750
00:25:10,400 –> 00:25:12,560
Lineage shows source event.

751
00:25:12,560 –> 00:25:13,840
Silver state.

752
00:25:13,840 –> 00:25:14,880
Gold measure.

753
00:25:14,880 –> 00:25:16,000
Reflex trigger.

754
00:25:16,000 –> 00:25:16,880
Case closed.

755
00:25:16,880 –> 00:25:18,400
Now HR.

756
00:25:18,400 –> 00:25:21,520
The domain everyone tiptoes around and leaks anyway.

757
00:25:21,520 –> 00:25:24,080
We isolate with workspaces and labels.

758
00:25:24,080 –> 00:25:26,880
NWHR PRD runs in trusted mode.

759
00:25:26,880 –> 00:25:30,400
Per view marks PII with sensitivity labels.

760
00:25:30,400 –> 00:25:31,360
Confidential.

761
00:25:31,360 –> 00:25:32,960
Highly confidential.

762
00:25:32,960 –> 00:25:37,360
Bronzelands payroll extracts and ATS applicant feeds.

763
00:25:37,360 –> 00:25:38,320
Immutable.

764
00:25:38,320 –> 00:25:40,240
Partitioned by arrival.

765
00:25:40,240 –> 00:25:43,840
Silver enforces schema, masks national identifiers

766
00:25:43,840 –> 00:25:45,840
into a quarantine reference.

767
00:25:45,840 –> 00:25:48,000
And emits conformed tables.

768
00:25:48,000 –> 00:25:50,960
Employees, positions, comp bands,

769
00:25:50,960 –> 00:25:52,720
headcount snapshots.

770
00:25:52,720 –> 00:25:56,080
Test block publish if referential integrity breaks.

771
00:25:56,080 –> 00:25:58,080
No business logic beyond conformance.

772
00:25:58,080 –> 00:25:59,760
Gold is two models, not one.

773
00:25:59,760 –> 00:26:03,040
An HR restricted semantic model with full PII

774
00:26:03,040 –> 00:26:04,880
under strict entry groups.

775
00:26:04,880 –> 00:26:07,840
An HR public certified with masked fields

776
00:26:07,840 –> 00:26:09,280
and only allowed aggregates.

777
00:26:09,280 –> 00:26:12,160
Direct lake reads from the same gold tables.

778
00:26:12,160 –> 00:26:15,600
RLS enforces geography and managerial hierarchy.

779
00:26:15,600 –> 00:26:17,360
Copilot assists in decks.

780
00:26:17,360 –> 00:26:19,920
But measures are defined in a metrics table,

781
00:26:19,920 –> 00:26:21,760
reviewed and certified.

782
00:26:21,760 –> 00:26:24,880
Audit logs prove who opened which report.

783
00:26:24,880 –> 00:26:27,840
If a contractor’s account tries to pivot on salary,

784
00:26:27,840 –> 00:26:29,600
RLS denies.

785
00:26:29,600 –> 00:26:32,320
And the event lands in the central audit.

786
00:26:32,320 –> 00:26:34,160
No exports to someone’s laptop.

787
00:26:34,160 –> 00:26:35,760
No personal tokens.

788
00:26:35,760 –> 00:26:37,840
The access path is the policy.

789
00:26:37,840 –> 00:26:39,280
Supply chain.

790
00:26:39,280 –> 00:26:41,760
Where latency costs money.

791
00:26:41,760 –> 00:26:46,240
Sources, ERP orders, WMS inventory, carrier scans,

792
00:26:46,240 –> 00:26:47,600
vendor ASNs.

793
00:26:47,600 –> 00:26:49,520
Bronzelands each feed as is.

794
00:26:49,520 –> 00:26:53,120
Silver conforms units, currencies, SKUs and time zones.

795
00:26:53,120 –> 00:26:56,640
We compute a canonical order silver, inventory position

796
00:26:56,640 –> 00:26:59,520
and a shipment events stream table pair.

797
00:26:59,520 –> 00:27:01,920
Deduplication is deterministic.

798
00:27:01,920 –> 00:27:05,680
Composite business keys plus latest event keys.

799
00:27:05,680 –> 00:27:09,200
A late arriving scan updates the state, not the past.

800
00:27:09,200 –> 00:27:11,040
Gold serves two models.

801
00:27:11,040 –> 00:27:15,120
An execution model, open orders, back orders, ATP,

802
00:27:15,120 –> 00:27:17,360
projected stockouts by node,

803
00:27:17,360 –> 00:27:20,720
with measures built for our level freshness.

804
00:27:20,720 –> 00:27:22,160
And a planning model.

805
00:27:22,160 –> 00:27:25,120
Weekly aggregates, lead time distributions,

806
00:27:25,120 –> 00:27:29,440
supplier, OTIF, forecast accuracy.

807
00:27:29,440 –> 00:27:34,160
Directly keeps execution fast without import cycles.

808
00:27:34,160 –> 00:27:35,680
Pipelines orchestrate.

809
00:27:35,680 –> 00:27:39,120
Event updates, trigger, silver microbatches.

810
00:27:39,120 –> 00:27:42,560
Scheduled windows, rebuild planning aggregates.

811
00:27:42,560 –> 00:27:46,480
Semantic models, reframe on partition completion.

812
00:27:46,480 –> 00:27:49,680
A stockout risk reflex posts into teams

813
00:27:49,680 –> 00:27:54,160
when ATP drops below threshold within lead time windows,

814
00:27:54,160 –> 00:27:57,040
tagging the correct regional planner.

815
00:27:57,040 –> 00:27:59,120
Lineage binds it all.

816
00:27:59,120 –> 00:28:03,680
Vendor ASN, row to stockout alert in two clicks.

817
00:28:03,680 –> 00:28:05,760
Three domains, same pattern.

818
00:28:05,760 –> 00:28:08,880
One lake, medallion, direct lake, purview,

819
00:28:08,880 –> 00:28:10,760
entra monitoring.

820
00:28:10,760 –> 00:28:12,400
No detours, no secrets.

821
00:28:12,400 –> 00:28:14,400
The edge cases don’t break the platform

822
00:28:14,400 –> 00:28:18,320
when the access path is real and the contracts are enforced.

823
00:28:18,320 –> 00:28:21,760
Hybrid walkthrough, one lake and purview security.

824
00:28:21,760 –> 00:28:24,560
Hybrid narration plus UI callouts.

825
00:28:24,560 –> 00:28:26,640
You want proof this isn’t theater.

826
00:28:26,640 –> 00:28:29,280
Fine, watch the access path form under your cursor.

827
00:28:29,280 –> 00:28:32,720
Open fabric left rail, workspaces, NW-plat.pl.D.

828
00:28:32,720 –> 00:28:34,200
This is the platform spine.

829
00:28:34,200 –> 00:28:38,120
Click, lake house, NW-plat.pl.D. Core Lake.

830
00:28:38,120 –> 00:28:40,320
Top right settings, see it.

831
00:28:40,320 –> 00:28:42,320
Trusted workspace access on.

832
00:28:42,320 –> 00:28:45,600
Network isolation configured to your policy.

833
00:28:45,600 –> 00:28:46,680
Why this matters?

834
00:28:46,680 –> 00:28:49,720
You don’t beg five products to honor your perimeter.

835
00:28:49,720 –> 00:28:52,840
One switch, one blast radius.

836
00:28:52,840 –> 00:28:54,480
Back to the lake house.

837
00:28:54,480 –> 00:28:57,120
One lake data hub, pane on the left.

838
00:28:57,120 –> 00:28:59,160
Files area, bronze.

839
00:28:59,160 –> 00:29:01,840
Tables area, silver and conformed dims.

840
00:29:01,840 –> 00:29:03,560
Expand files, bronze sources,

841
00:29:03,560 –> 00:29:06,600
Epoorders year 2025, month 11.

842
00:29:06,600 –> 00:29:08,800
Immutable, partitioned by arrival.

843
00:29:08,800 –> 00:29:12,680
You’re not cleaning, you’re preserving evidence.

844
00:29:12,680 –> 00:29:15,560
Now tables, silver.orders.

845
00:29:15,560 –> 00:29:18,200
Click the table, schema view.

846
00:29:18,200 –> 00:29:21,880
Types explicit, nullability declared.

847
00:29:21,880 –> 00:29:24,200
Partitioning by order date.

848
00:29:24,200 –> 00:29:27,920
Properties panel, sensitivity shows internal.

849
00:29:27,920 –> 00:29:29,200
Per view will read this.

850
00:29:29,200 –> 00:29:30,320
It will travel.

851
00:29:30,320 –> 00:29:32,960
You won’t be able to accidentally launder PII

852
00:29:32,960 –> 00:29:35,760
into a public report and claim you didn’t know.

853
00:29:35,760 –> 00:29:37,760
The label follows the asset.

854
00:29:37,760 –> 00:29:40,320
UI call out, new shortcut.

855
00:29:40,320 –> 00:29:41,200
Click it.

856
00:29:41,200 –> 00:29:42,920
Choose external one lake.

857
00:29:42,920 –> 00:29:46,440
Browse to NW sales PRDLH.

858
00:29:46,440 –> 00:29:48,480
Tables, silver.

859
00:29:48,480 –> 00:29:49,680
Customers.

860
00:29:49,680 –> 00:29:51,680
Add, no copy, no drift.

861
00:29:51,680 –> 00:29:54,080
Security posture inherited from source.

862
00:29:54,080 –> 00:29:56,320
Shortcuts are your anti-duplication weapon.

863
00:29:56,320 –> 00:29:58,000
You’re allowed references.

864
00:29:58,000 –> 00:30:00,000
You’re not allowed shadow clones.

865
00:30:00,000 –> 00:30:04,880
Security, top bar, ellipses, manage access.

866
00:30:04,880 –> 00:30:08,560
Workspace roles, admins, platform engineering only.

867
00:30:08,560 –> 00:30:10,640
Members restricted, contributors,

868
00:30:10,640 –> 00:30:14,480
none outside the platform, viewers, service principles tied

869
00:30:14,480 –> 00:30:16,120
to pipeline runs.

870
00:30:16,120 –> 00:30:19,760
Principle of least privilege and forced at the workspace

871
00:30:19,760 –> 00:30:22,040
not negotiated per item.

872
00:30:22,040 –> 00:30:24,440
If someone asks for just this one table,

873
00:30:24,440 –> 00:30:26,360
the answer is still no.

874
00:30:26,360 –> 00:30:28,760
It’s a boundary, not a suggestion, identity,

875
00:30:28,760 –> 00:30:31,920
enter groups, click add people or groups, type NW

876
00:30:31,920 –> 00:30:33,640
Plot PRD admins.

877
00:30:33,640 –> 00:30:35,640
It resolves to your enter security group,

878
00:30:35,640 –> 00:30:37,680
same for members, readers.

879
00:30:37,680 –> 00:30:39,800
This is the sentence you’ve been missing.

880
00:30:39,800 –> 00:30:42,240
Access to platform data is governed by

881
00:30:42,240 –> 00:30:45,680
enter groups bound to workspace roles.

882
00:30:45,680 –> 00:30:49,840
No personal tokens, no local users, no exceptions.

883
00:30:49,840 –> 00:30:53,840
Now governance, left rail, purview hub, open it,

884
00:30:53,840 –> 00:30:59,640
sources, register one lake item, NW Plot PRD Core Lake.

885
00:30:59,640 –> 00:31:03,360
Choose scan rule set, standard plus sensitive info.

886
00:31:03,360 –> 00:31:07,360
Schedule nightly to AM before business opens.

887
00:31:07,360 –> 00:31:08,920
Click save and run.

888
00:31:08,920 –> 00:31:11,760
This is where you stop drawing lineage in PowerPoint

889
00:31:11,760 –> 00:31:14,600
once the scan finishes open data map.

890
00:31:14,600 –> 00:31:16,240
Search for silver orders.

891
00:31:16,240 –> 00:31:17,160
Click it.

892
00:31:17,160 –> 00:31:18,840
See the asset page.

893
00:31:18,840 –> 00:31:22,720
Properties, owner, description, business glossary terms.

894
00:31:22,720 –> 00:31:27,280
Assign terms, order, fulfillment, cancellation reason.

895
00:31:27,280 –> 00:31:28,800
Now attach a steward.

896
00:31:28,800 –> 00:31:30,360
Now click lineage.

897
00:31:30,360 –> 00:31:34,400
This is the part your old stack couldn’t show without lies.

898
00:31:34,400 –> 00:31:35,680
Upstream.

899
00:31:35,680 –> 00:31:39,200
Ingest pipeline activity, bronze.

900
00:31:39,200 –> 00:31:43,800
ERP orders partition, transform node,

901
00:31:43,800 –> 00:31:49,320
notebook, promote, bronze to silver, with run IDs.

902
00:31:49,320 –> 00:31:55,840
Downstream, gold.f orders, gold.d customer via shortcut,

903
00:31:55,840 –> 00:32:01,080
semantic model, NW sales PRD gold sales ops reports

904
00:32:01,080 –> 00:32:06,320
that consume it, each hop timestamped, each dependency real.

905
00:32:06,320 –> 00:32:09,600
When an exec asks where did this number come from?

906
00:32:09,600 –> 00:32:10,560
You don’t perform.

907
00:32:10,560 –> 00:32:12,440
You show sensitivity.

908
00:32:12,440 –> 00:32:13,960
Back to the asset page.

909
00:32:13,960 –> 00:32:14,700
Edit.

910
00:32:14,700 –> 00:32:17,480
Set sensitivity, confidential for silver.

911
00:32:17,480 –> 00:32:19,040
Orders, save.

912
00:32:19,040 –> 00:32:21,200
Now jump to Power BI service.

913
00:32:21,200 –> 00:32:23,120
Shared analytics workspace.

914
00:32:23,120 –> 00:32:25,880
Open the sales certified semantic model.

915
00:32:25,880 –> 00:32:27,960
Settings, sensitivity.

916
00:32:27,960 –> 00:32:31,600
See it inherit the highest label from upstream tables.

917
00:32:31,600 –> 00:32:32,600
You didn’t relabel.

918
00:32:32,600 –> 00:32:33,720
It propagated.

919
00:32:33,720 –> 00:32:35,040
Open the sales report.

920
00:32:35,040 –> 00:32:37,360
The label is on the report artifact too.

921
00:32:37,360 –> 00:32:40,920
Export to Excel governed by M365 sensitivity rules.

922
00:32:40,920 –> 00:32:44,920
Screen shots won’t save you, but X filtration gets harder.

923
00:32:44,920 –> 00:32:47,760
And auditable UI call out, purview policy.

924
00:32:47,760 –> 00:32:51,280
Back in purview hub policies, create data access policy.

925
00:32:51,280 –> 00:32:56,400
Scope, NW, PlatPRD core lake table, silver.

926
00:32:56,400 –> 00:32:59,880
Allow red for Entra group and undo sales PRD readers.

927
00:32:59,880 –> 00:33:02,200
Deny data actions for external users.

928
00:33:02,200 –> 00:33:02,720
Publish.

929
00:33:02,720 –> 00:33:05,160
You didn’t touch ACLs in five places.

930
00:33:05,160 –> 00:33:07,720
You set an access intent in one governance plane.

931
00:33:07,720 –> 00:33:09,120
The platform enforces it.

932
00:33:09,120 –> 00:33:13,720
Monitoring the posture, fabric monitoring hub, filters,

933
00:33:13,720 –> 00:33:19,840
workspace, it’s NW plate PRD, tabs, pipelines, notebooks,

934
00:33:19,840 –> 00:33:24,360
Power BI, warehouse, capacity, flip to auditing.

935
00:33:24,360 –> 00:33:29,480
Access logs show service principle NW Platy PRD pipelines

936
00:33:29,480 –> 00:33:34,480
reading bronze ERP orders at 0105 human access to silver.

937
00:33:34,480 –> 00:33:38,800
Orders at 0917 by a member report open events at 0922

938
00:33:38,800 –> 00:33:39,800
correlate.

939
00:33:39,800 –> 00:33:45,040
If an impossible value shows up, you can prove who touched what when.

940
00:33:45,040 –> 00:33:49,120
Now force a failure ingest a malformed file into bronze.

941
00:33:49,120 –> 00:33:52,200
Earp orders data factory pipeline and W PlatPRD

942
00:33:52,200 –> 00:33:53,560
ingest moves it in.

943
00:33:53,560 –> 00:33:56,760
Promote bronze to silver notebook runs and rejects

944
00:33:56,760 –> 00:33:58,280
rows to silver.

945
00:33:58,280 –> 00:34:00,320
Orders quarantine with reasons.

946
00:34:00,320 –> 00:34:03,560
Per view scan runs at Ero2 Huntook and discovers a rise

947
00:34:03,560 –> 00:34:05,200
in rejected rows.

948
00:34:05,200 –> 00:34:06,720
Trigger and alert.

949
00:34:06,720 –> 00:34:09,120
In monitoring, you see the pipeline went red.

950
00:34:09,120 –> 00:34:11,920
Validation notebook flagged drift, semantic model

951
00:34:11,920 –> 00:34:15,680
reframing paused because gold partition completion rule

952
00:34:15,680 –> 00:34:17,600
didn’t meet threshold.

953
00:34:17,600 –> 00:34:19,960
Nothing mysteriously updated.

954
00:34:19,960 –> 00:34:23,720
The contracts held, the blast radius stopped at silver.

955
00:34:23,720 –> 00:34:25,280
That’s the point.

956
00:34:25,280 –> 00:34:27,240
Edge case you think beats this.

957
00:34:27,240 –> 00:34:28,640
External S3.

958
00:34:28,640 –> 00:34:31,480
Open NW PlatPRD Corlec new shortcut.

959
00:34:31,480 –> 00:34:33,040
Amazon S3.

960
00:34:33,040 –> 00:34:35,680
Enter bucket path to raw vendor feed.

961
00:34:35,680 –> 00:34:39,200
Authenticate with a managed identity or ses.

962
00:34:39,200 –> 00:34:41,760
Not a shared key baked into code.

963
00:34:41,760 –> 00:34:45,920
Add, bronze now references external data without copying.

964
00:34:45,920 –> 00:34:48,120
Per view still scans the reference table

965
00:34:48,120 –> 00:34:50,960
once it lands as delta in silver.

966
00:34:50,960 –> 00:34:52,400
Governance remains central.

967
00:34:52,400 –> 00:34:54,000
Identity remains entra.

968
00:34:54,000 –> 00:34:56,720
You didn’t break the model to appease an external vendor.

969
00:34:56,720 –> 00:34:57,520
One more lock.

970
00:34:57,520 –> 00:34:59,600
Row level security for consumption.

971
00:34:59,600 –> 00:35:02,320
In the sales semantic model, model view.

972
00:35:02,320 –> 00:35:06,960
Define RLS roles using DAX filters tied to entra groups.

973
00:35:06,960 –> 00:35:10,160
Region in values, user principle name.

974
00:35:10,160 –> 00:35:12,760
Mapped through a security bridge table.

975
00:35:12,760 –> 00:35:15,480
Assign entra groups in the data set security blade.

976
00:35:15,480 –> 00:35:16,960
This is consumption RLS.

977
00:35:16,960 –> 00:35:20,280
Workspace roles protect platform surfaces.

978
00:35:20,280 –> 00:35:23,160
Two layers, two purposes, no confusion.

979
00:35:23,160 –> 00:35:24,040
You see it now.

980
00:35:24,040 –> 00:35:25,800
One leg contains the data.

981
00:35:25,800 –> 00:35:27,440
Workspaces contain the humans.

982
00:35:27,440 –> 00:35:28,880
Entra contains the identity.

983
00:35:28,880 –> 00:35:30,160
Per view contains the map.

984
00:35:30,160 –> 00:35:33,000
Monitoring contains the truth of what happened.

985
00:35:33,000 –> 00:35:34,720
Shortcuts cut copies out of your life.

986
00:35:34,720 –> 00:35:37,680
Sensitivity labels travel so you can’t pretend data

987
00:35:37,680 –> 00:35:40,840
changed its nature when it hits a pretty chart.

988
00:35:40,840 –> 00:35:44,040
If you can’t show this walk on your platform today,

989
00:35:44,040 –> 00:35:48,760
identity to item, lineage to report, label to export,

990
00:35:48,760 –> 00:35:51,600
failure to alert, you don’t have governance.

991
00:35:51,600 –> 00:35:54,440
You have a spreadsheet that says we tried.

992
00:35:54,440 –> 00:35:56,440
Fabric makes the policy the path.

993
00:35:56,440 –> 00:35:59,560
Now the only question is whether you follow it.

994
00:35:59,560 –> 00:36:01,040
Start fabric the right way.

995
00:36:01,040 –> 00:36:02,480
Do this in seven days.

996
00:36:02,480 –> 00:36:04,600
You don’t fix entropy with vibes.

997
00:36:04,600 –> 00:36:07,160
You fix it with a week of disciplined moves.

998
00:36:07,160 –> 00:36:08,440
Seven days.

999
00:36:08,440 –> 00:36:09,720
No heroics.

1000
00:36:09,720 –> 00:36:13,720
Just the minimum viable platform that refuses to lie.

1001
00:36:13,720 –> 00:36:14,720
Day one.

1002
00:36:14,720 –> 00:36:16,920
Capacity, identity, boundaries.

1003
00:36:16,920 –> 00:36:19,840
In Azure, deploy a fabric FSQ.

1004
00:36:19,840 –> 00:36:20,680
Tag it.

1005
00:36:20,680 –> 00:36:21,680
ENVia prod.

1006
00:36:21,680 –> 00:36:22,960
Owner but platform.

1007
00:36:22,960 –> 00:36:23,960
Cost Center.

1008
00:36:23,960 –> 00:36:24,960
Analytics.

1009
00:36:24,960 –> 00:36:30,120
Map budgets in Azure cost management with alerts at 70, 90, 100%.

1010
00:36:30,120 –> 00:36:32,800
In Microsoft, 365.

1011
00:36:32,800 –> 00:36:34,040
Confirm Entra.

1012
00:36:34,040 –> 00:36:35,360
Conditional access.

1013
00:36:35,360 –> 00:36:36,360
Baseline.

1014
00:36:36,360 –> 00:36:37,880
MFA required.

1015
00:36:37,880 –> 00:36:39,680
Compliant device for admins.

1016
00:36:39,680 –> 00:36:42,600
Sign in risk policy enabled.

1017
00:36:42,600 –> 00:36:47,400
Create Entra groups that match workspace roles before you create the workspaces.

1018
00:36:47,400 –> 00:36:50,920
NW, Plat, PR, the admins.

1019
00:36:50,920 –> 00:36:52,480
Members, readers.

1020
00:36:52,480 –> 00:36:54,400
Same pattern for dev and test.

1021
00:36:54,400 –> 00:36:55,560
Same for each domain.

1022
00:36:55,560 –> 00:36:59,120
NW sales PR, the admins, members, readers.

1023
00:36:59,120 –> 00:37:00,920
Names are boring on purpose.

1024
00:37:00,920 –> 00:37:02,240
Boring scales.

1025
00:37:02,240 –> 00:37:03,600
Day two.

1026
00:37:03,600 –> 00:37:06,920
Workspaces and one lake spine.

1027
00:37:06,920 –> 00:37:09,400
Create three platform workspaces.

1028
00:37:09,400 –> 00:37:14,400
NW, Plat, dev, NW, Plat, TST, NW, Plat, PR, D.

1029
00:37:14,400 –> 00:37:16,760
Assign them to your fabric capacity.

1030
00:37:16,760 –> 00:37:20,320
In each create a core lake house and a core warehouse.

1031
00:37:20,320 –> 00:37:24,200
In the core lake house define bronze file structure first.

1032
00:37:24,200 –> 00:37:29,440
Files, bronze, sources, system, entity, partition.

1033
00:37:29,440 –> 00:37:31,440
Partition by arrival cadence.

1034
00:37:31,440 –> 00:37:36,440
Create table silver with a placeholder empty schema for your first entities.

1035
00:37:36,440 –> 00:37:39,520
Builders, customers, products, delta format.

1036
00:37:39,520 –> 00:37:45,840
In workspace settings enable trusted workspace mode and network isolation per your perimeter.

1037
00:37:45,840 –> 00:37:48,920
Bind the Entra groups to workspace roles.

1038
00:37:48,920 –> 00:37:50,240
No individuals.

1039
00:37:50,240 –> 00:37:51,720
No exceptions.

1040
00:37:51,720 –> 00:37:52,800
Day three.

1041
00:37:52,800 –> 00:37:55,160
In jest and validation contracts.

1042
00:37:55,160 –> 00:38:00,160
In NW, Plat, dev build one data flow, gen two or pipeline for a single source entity,

1043
00:38:00,160 –> 00:38:02,160
ERP orders.

1044
00:38:02,160 –> 00:38:08,120
Create raw into bronze with metadata columns, arrival T’s, source, correlation ID.

1045
00:38:08,120 –> 00:38:09,520
Build one notebook.

1046
00:38:09,520 –> 00:38:11,560
Promote bronze to silver orders.

1047
00:38:11,560 –> 00:38:17,320
It reads bronze profile schema and forces types dedupes deterministically write silver

1048
00:38:17,320 –> 00:38:20,120
with partitioning by business date.

1049
00:38:20,120 –> 00:38:23,360
Add a quarantine table for rejects with reasons.

1050
00:38:23,360 –> 00:38:25,040
Write three tests.

1051
00:38:25,040 –> 00:38:28,120
Row count sanity primary key uniqueness.

1052
00:38:28,120 –> 00:38:31,280
Foreign key existence to a conformed dim.

1053
00:38:31,280 –> 00:38:32,880
Double it if needed.

1054
00:38:32,880 –> 00:38:35,760
Fail the pipeline if tests fail.

1055
00:38:35,760 –> 00:38:37,880
Identit write only.

1056
00:38:37,880 –> 00:38:39,600
Atomic commits.

1057
00:38:39,600 –> 00:38:44,000
You’re teaching the system to refuse bad truth.

1058
00:38:44,000 –> 00:38:45,480
Day four.

1059
00:38:45,480 –> 00:38:48,480
Gold and a semantic model without lies.

1060
00:38:48,480 –> 00:38:50,320
Create a gold table.

1061
00:38:50,320 –> 00:38:52,960
F orders from silver.

1062
00:38:52,960 –> 00:38:55,400
Orders with stable grain.

1063
00:38:55,400 –> 00:39:01,080
Create conformed dims for date, customer, product in platform silver.

1064
00:39:01,080 –> 00:39:04,200
Orders them to domains via shortcuts.

1065
00:39:04,200 –> 00:39:09,040
In shared analytics dev build one semantic model in direct lake over gold F orders and

1066
00:39:09,040 –> 00:39:10,360
dims.

1067
00:39:10,360 –> 00:39:13,920
Define five certified measures in a metrics table.

1068
00:39:13,920 –> 00:39:19,160
Orders, revenue, margin, average order value, on time percent if you have it.

1069
00:39:19,160 –> 00:39:21,160
No kpi logic buried upstream.

1070
00:39:21,160 –> 00:39:24,800
Wire a single report page with those measures in a date slicer.

1071
00:39:24,800 –> 00:39:26,240
It’s not a dashboard.

1072
00:39:26,240 –> 00:39:27,240
It’s a proof.

1073
00:39:27,240 –> 00:39:31,640
Direct lake path works measures are index and no copies exist.

1074
00:39:31,640 –> 00:39:32,960
Day five.

1075
00:39:32,960 –> 00:39:34,920
Governance stitched into the path.

1076
00:39:34,920 –> 00:39:40,760
Open purview hub register NW Platt PRD core lake house and warehouse.

1077
00:39:40,760 –> 00:39:45,520
Configure a scan rule set that includes sensitive info detection.

1078
00:39:45,520 –> 00:39:47,120
Schedule nightly scans.

1079
00:39:47,120 –> 00:39:51,760
Assign business glossary terms to silver assets.

1080
00:39:51,760 –> 00:39:52,760
Order.

1081
00:39:52,760 –> 00:39:53,760
Customer.

1082
00:39:53,760 –> 00:39:55,440
Cancelation reason.

1083
00:39:55,440 –> 00:40:01,320
Accessitivity on silver tables where appropriate internal confidential confirm propagation

1084
00:40:01,320 –> 00:40:06,160
into the shared analytics semantic model and report labels.

1085
00:40:06,160 –> 00:40:12,560
Create a purview data access policy that grants read to NW sales PRD readers on specific

1086
00:40:12,560 –> 00:40:17,440
silver tables publish in power bi set certification rules.

1087
00:40:17,440 –> 00:40:23,000
So only platform stewards can certify data sets and only shared analytics can host certified

1088
00:40:23,000 –> 00:40:24,080
models.

1089
00:40:24,080 –> 00:40:27,320
You just turned governance from a memo into code.

1090
00:40:27,320 –> 00:40:33,320
Day six monitoring failure drills reframing discipline in monitoring hub pin of view filter

1091
00:40:33,320 –> 00:40:41,840
to NW Platt PRD and shared analytics PRD pipelines notebooks power bi capacity add alerts

1092
00:40:41,840 –> 00:40:50,400
for pipeline failures and capacity saturation 80% for 15 minutes in the semantic model settings

1093
00:40:50,400 –> 00:40:57,600
disable auto detect reframing for production use pipeline driven reframing after gold partition

1094
00:40:57,600 –> 00:40:58,600
completion.

1095
00:40:58,600 –> 00:41:03,280
Now break something on purpose push a malformed bronze file watch the notebook reject rose

1096
00:41:03,280 –> 00:41:10,160
to quarantine pipeline fail reframing pause verify lineage in purview shows the fail transform

1097
00:41:10,160 –> 00:41:15,720
fix the source rerun watch success propagate you’re teaching the team what loud failure looks

1098
00:41:15,720 –> 00:41:18,320
like silences the enemy.

1099
00:41:18,320 –> 00:41:24,840
Day seven domains shortcuts and access path rehearsal create one domain workspace set

1100
00:41:24,840 –> 00:41:33,780
NW sales dev TST PRD in NW sales PRD create a lake house add shortcuts to platform silver

1101
00:41:33,780 –> 00:41:41,840
orders customers products build one domain specific goal table F sales ops derived from platform

1102
00:41:41,840 –> 00:41:48,600
silver no duplication no upstream logic build a domain semantic model apply RLS using

1103
00:41:48,600 –> 00:41:53,800
intro groups mapped through a security bridge table publish a thin report to shared analytics

1104
00:41:53,800 –> 00:41:59,760
PRD and certified after review now rehearse the access path with your stakeholders show

1105
00:41:59,760 –> 00:42:06,000
one lake folder for bronze silver table schema goal table contract semantic model measures

1106
00:42:06,000 –> 00:42:12,480
lineage graph sensitivity label on the report and monitoring hub for the last 24 hours name

1107
00:42:12,480 –> 00:42:16,760
each boundary allowed if there’s a step you can’t show fix it the platform is the demo

1108
00:42:16,760 –> 00:42:22,880
at the end of seven days you don’t have everything you have the spine one capacity one platform

1109
00:42:22,880 –> 00:42:30,600
one domain one end to end path contracts and forced governance visible monitoring loud

1110
00:42:30,600 –> 00:42:36,520
you replaced we’re working on it with evidence now you scale by repetition not reinvention

1111
00:42:36,520 –> 00:42:42,360
conclusion the fabric mindset shift here’s the shift you stop treating data as a project

1112
00:42:42,360 –> 00:42:48,800
you treat it as a system with one access path one set of contracts one blast radius bronze

1113
00:42:48,800 –> 00:42:56,040
preserves evidence silver and forces truth gold exposes meaning workspaces contain humans

1114
00:42:56,040 –> 00:43:02,560
one lake contains data entra contains identity purview contains the map monitoring contains

1115
00:43:02,560 –> 00:43:07,800
the heartbeat if you’re ready to build the spine in seven days do it now subscribe then

1116
00:43:07,800 –> 00:43:12,720
watch the dev test prod deployment playbook next it’s the exact pipeline and gede flow

1117
00:43:12,720 –> 00:43:19,040
I use to enforce these contracts at scale and the theater name your access path enforce it

1118
00:43:19,040 –> 00:43:21,680
or admit you’re still exporting to excel in the dark