This article is for organizations looking to migrate more than 400,000 files from a third-party system into Google Drive. Before doing a large migration, review these best practices. They establish a baseline for how to approach a migration and create a folder and document hierarchy for a large number of files. These guidelines are starting points for organizing your Drive. Expect and plan for organic growth of your content over time. The guidance below helps you get started without performance concerns, with room to grow as an organization.
Before you begin
For most large migrations:
- You can use Google Workspace Migrate.
To compare migration products to see which one is best for you, see the Google Workspace migration product matrix.
- We recommend you work with a Google partner.
File creation and ownership
To reduce the rate of errors and improve performance both during and after migration, do not let a single user create or own more than 400,000 files, folders, or shortcuts.
To expedite migration, instead use service accounts to shard file creation across multiple accounts. Service accounts are machine accounts that perform non-person business functions throughout Google Workspace.
If folder hierarchies grow large, updating them can take a long time. To facilitate the updating of permissions and items, avoid putting too many items in a single folder or nesting hierarchies deeply. Avoid nesting more than 20 levels of folders in My Drive or a shared drive. See Shared drive limits.
Any one folder is limited to 500,000 items directly in that folder, so limit the items in a folder to 400,000 to start. Learn more about Folder limits in Drive.
Use shared drives
Files are created by a user and then can be added to a shared drive, at which point the shared drive becomes the owner. Put items that don’t need to belong to one person in a shared drive, and you can also add specific groups as members of the shared drive. Each shared drive can hold 400,000 items.
If members leave your organization, the files in shared drives stay in place. Your team can keep sharing information, working anywhere, from any device. For more information, see What are shared drives?
Sharing and permissions in Google Drive
Inheritance is a critical concept for understanding file sharing in Drive and for successful migration. Files inherit the sharing controls of folders they’re in. So, you should usually apply permissions to folders before adding files.
Upload limits, quotas, and exemptions
Other system limits protect your data and ensure performance. However, you can request exemptions from them for your migration. See Upload files to Google Drive.
- Batch multiple calls into a single request using the Google Drive API, especially when changing file metadata for many items that belong to the same user. See Google Drive API.
- Do not allow a service account to create more than 400,000 files on behalf of a given user account.
- The default quota limits for Drive API are 10,000 calls every 100 seconds, both per user and per project. This limit applies to the sum of read and write calls. If you think you might exceed these quotas, request an increase. See Monitor API quotas.
- The rate of Drive API write requests is limited—avoid exceeding 3 requests per second of sustained write or insert requests, per account. Note: This rate limit can’t be increased.
- If you encounter errors during the migration, follow steps to resolve them. See Retry failed requests to resolve errors.
Use service accounts to impersonate many users & upload files
Important: Only use service accounts for delegation where the identity is that of an individual user. Using the service account as a common owner to create many shared documents can create performance problems. Do not use service accounts to acquire additional storage quota or act as members of a domain.
Service accounts play a critical role in migrations because they automate content creation on behalf of users, applying permissions and placing items appropriately in your folder structure. Using service accounts can help you complete your migration faster by spreading the migration load across many users.
Use them to:
- Create content on behalf of a user by having them impersonate the user.
- Spread out upload and insertion requests across many user accounts. To speed up uploading content to Google Drive and avoid quota issues, use a service account with domain-wide delegation to distribute write activity across many different impersonated users.
Learn how to:
Note: Instead of assigning ownership of files and folders to the service account performing the upload, place the content in a shared drive or assign ownership to a user account.
Before the migration, design a system to organize items in your body of content. For example, create file metadata and structure your organization’s content by taking these steps:
- Apply custom file properties. Learn more
- Restrict authoritative versions of files with locking. Learn more
- Apply Google Vault retention policies to your content. Learn more
- Specify your organization’s sharing policies. Learn more
Still need help?
Contact your migration partner. If you’re not working with a partner, contact Google Workspace support.