OneLake Shortcuts Explained
Access data across Azure, AWS, and GCP storage without moving or copying it using OneLake shortcuts. Unified data access for Microsoft Fabric workloads.
OneLake shortcuts are one of Microsoft Fabric's most powerful features for breaking down data silos without the cost, latency, and complexity of traditional data movement pipelines. A shortcut creates a virtual reference to data stored outside your Lakehouse—in Azure Data Lake Storage, Amazon S3, Google Cloud Storage, or another Fabric workspace—making it appear as a native Lakehouse table or folder. Spark notebooks, SQL queries, dataflows, and Direct Lake semantic models all access shortcut data identically to locally stored data. The result is a unified analytics layer across your entire data estate without duplicating a single byte.
How Shortcuts Work
Architecture
A shortcut is a metadata pointer stored in OneLake that redirects read requests to the actual data location. When a Spark notebook or SQL query references a shortcut table, the Fabric compute engine reads data directly from the source storage through the configured authentication credentials. The data never passes through an intermediate copy—reads go straight to the source.
Important Characteristics
| Feature | Behavior | Implication | |---|---|---| | Read/Write | Read-only for external sources; read-write for OneLake-to-OneLake | Cannot modify S3 or ADLS data through shortcuts | | Latency | Depends on source storage performance and network | Cross-region shortcuts may have higher latency | | Authentication | Per-shortcut credentials (SAS, key, service principal, OAuth) | Credentials must be maintained and rotated | | Format support | Delta, Parquet, CSV (Files section only) | Tables section requires Delta format | | Cost | No OneLake storage cost; data egress from source may apply | S3 and ADLS egress charges still apply | | Caching | OneLake may cache frequently accessed data | Reduces latency for repeated reads |
Tables vs Files Shortcuts
Shortcuts can be created in two locations within a Lakehouse:
- Tables section: The shortcut target must contain Delta-formatted data. The data appears as a queryable table with schema, column types, and support for SQL and Spark DataFrame operations.
- Files section: The shortcut target can contain any file format (Parquet, CSV, JSON, images, etc.). Files appear in the Lakehouse file browser but are not automatically queryable as tables—you must read them explicitly in Spark or register them as external tables.
Supported Data Sources
Azure Data Lake Storage Gen2 (ADLS)
The most common shortcut source for organizations with existing Azure data lakes:
- Authentication options: Account key, SAS token, or service principal (OAuth). Service principal is recommended for production—it supports credential rotation and RBAC without sharing storage keys.
- Path configuration: Specify the storage account URL, container name, and folder path. You can shortcut to a specific folder within a container, not just the root.
- DFS endpoint: Use the `dfs.core.windows.net` endpoint (not `blob.core.windows.net`) for optimal performance.
- Hierarchical namespace: ADLS Gen2 with hierarchical namespace enabled is required. Standard Blob Storage without HNS is not supported.
Amazon S3
Access data in AWS without moving it to Azure:
- Authentication: S3 access key ID and secret access key. Store credentials securely—Fabric encrypts them at rest but plan for regular rotation.
- Path configuration: Provide the S3 bucket name and prefix (folder path). Shortcuts work with standard S3 and S3-compatible storage (MinIO, Wasabi).
- Cross-cloud performance: Data reads traverse the internet between AWS and Azure regions. For large datasets, select AWS regions geographically close to your Fabric capacity region to minimize latency. Cross-cloud egress charges apply.
- Partitioned data: If S3 data is Hive-partitioned (e.g., /year=2024/month=01/), Spark can leverage partition pruning through shortcuts just as with local data.
Google Cloud Storage
Access data in GCP through S3-compatible API:
- Authentication: HMAC keys (interoperability credentials) that provide S3-compatible access to GCS buckets
- Configuration: Use the GCS S3-compatible endpoint with your HMAC credentials
- Same cross-cloud considerations as S3: Latency and egress costs apply
OneLake (Internal Fabric)
Shortcuts between Fabric workspaces enable data sharing without duplication:
- Cross-workspace: Reference a Lakehouse table in another workspace. Users need at least read permissions on the source workspace.
- Cross-tenant: External data shares enable OneLake shortcuts across Fabric tenants—useful for partner organizations sharing data securely.
- Read-write: Unlike external shortcuts, OneLake-to-OneLake shortcuts support write operations if the user has write permissions on the source.
Use Cases and Patterns
Pattern 1: Multi-Cloud Data Unification
An organization stores transactional data in Azure SQL (mirrored to OneLake), clickstream data in S3, and IoT telemetry in GCS. Create shortcuts in a single "Unified Analytics" Lakehouse to access all three sources. Build Spark notebooks and Direct Lake semantic models that join across all sources without any data movement.
Pattern 2: Department-Level Data Products
A central data engineering team maintains curated Gold layer tables in a "Core Data" Lakehouse. Department-specific Lakehouses (Finance, Marketing, HR) create shortcuts to the Gold tables they need, then add department-specific transformations and tables alongside the shortcuts. Each department gets a tailored workspace without duplicating core data.
Pattern 3: Legacy Data Lake Migration
Organizations migrating from an existing ADLS Gen2 data lake to Fabric can use shortcuts as a bridge. Create shortcuts to existing ADLS folders, immediately enabling Fabric SQL and Spark queries against legacy data. Migrate tables to native Lakehouse storage incrementally—replacing shortcuts with local tables as each table is validated.
Pattern 4: Real-Time External Data Access
For data that changes frequently in external storage (e.g., a partner's S3 bucket updated hourly), shortcuts provide automatic access to the latest version without scheduling copy pipelines. Every query reads the current state of the source data.
Security and Governance
Credential Management
- Service principals (recommended): Register an Entra ID application with read access to the target storage. Credentials can be rotated without updating individual shortcuts.
- SAS tokens: Time-limited and scope-limited. Good for temporary access or proof of concept but problematic at scale because tokens expire and must be regenerated.
- Account keys: Full access to the storage account. Not recommended for production—if compromised, all data in the account is exposed.
Access Control Through Shortcuts
OneLake enforces Fabric workspace-level security on shortcut access. A user must have at least Viewer role on the workspace containing the shortcut to read data through it. However, shortcuts do not add security to the source—if a user has direct access to the S3 bucket or ADLS account, they can access data independently of Fabric permissions.
Audit and Compliance
Shortcut access is logged in Fabric audit logs. For compliance requirements, track which shortcuts exist, what sources they reference, who created them, and when they were last accessed. Include shortcut inventory in your data governance documentation.
Performance Optimization
- Prefer same-region sources: Shortcuts to storage in the same Azure region as your Fabric capacity have the lowest latency
- Use Delta format: Tables section shortcuts require Delta format, which enables predicate pushdown and column pruning—dramatically reducing data read from source
- Partition pruning: Hive-partitioned data benefits from automatic partition pruning in Spark queries, reading only relevant partitions
- Cache warming: Frequently accessed shortcut data is cached in OneLake, improving performance on repeated queries
- Avoid cross-region for large tables: If a shortcut crosses regions and the table is large (>10GB), consider a one-time copy to local Lakehouse storage instead
Related Resources
Frequently Asked Questions
Does OneLake shortcuts copy the data?
No, shortcuts are pointers only. The data remains in its original location. OneLake queries the source directly, so you always access the current data without duplication.
Can I write data through a shortcut?
Shortcuts are read-only for external sources. To write data, you need to use the native Lakehouse tables or write directly to the external storage through its own APIs.