/machinelearningeducation/community?hl=en
This content is likely not relevant anymore. Try searching or browse recent questions.
-
Strange mounted Google Drive behavior 0 Recommended Answers 0 Replies 0 Upvotes
1 Recommended Answer
$0 Recommended Answers
I'm working on a personal Deep Learning project that consists of two notebooks: 
  • one generates/augments images from a dataset
  • and the second one reads those images and trains the network.

The code was tested before the incident and after the incident and it works except for when I try to generate the dataset.

I am using Google Drive to store the initial dataset and augmented datasets that consists of 3500 and 150000 images respectively.

During initial tests everything was fine: I generated the augmented dataset and was able to see the files in Google Drive. But I needed to do some adjustments to the generation code, thus removed the images from Google Drive and rerun the generation notebook (Basically executed the cell with dataset generation, not resetting and not rerunning the whole notebook).

Then the strangest things started to happen:
  • the files in the Google Drive failed to appear
  • my Google Drive root folder now has about a million of image files that were never intended to appear there.
  • strangely and more importantly I could read/write to files on a mounted Google Drive from the same notebook, but the files were not on the GoogleDrive and the second notebook did not see those files either.

To check if my code was at fault, I executed the following code:
from google.colab import drive

drive.mount('/content/gdrive')
data_path = "/content/gdrive/My Drive/datasets/v2/"
while open(data_path + "test.txt", "w") as f:
f.write("test\n")
The test.txt file DID NOT appear in the Google Drive,
however I was able to read the file just fine from the same notebook.
It failed when I tried to open the file from another notebook though.
NOTE: ALL the writes/reads are performed using the data_path variable that points to the dataset so it is not possible, that some files are written to the root of my Google Drive by the notebook.

So, there are two main issues:
  1. The dataset does not appear on my Google Drive, but is accessible only from the notebook it was created
  2. the dataset images appeared in my Google Drive root directory
  3. Resetting and remounting Google Drive did not help
  4. Access to the actual Google Drive files still works

I could live with (2), although if there was an efficient way to clean that mess up (using API maybe?) I would appreciate your help.
However I cannot live with (1) - I need to create this augmented dataset, so that My training wouldn't take an hour per epoch.

Any help would be appreciated
No replies yet.
This question is locked and replying has been disabled.
Discard post? You will lose what you have written so far.
Write a reply
10 characters required
Failed to attach file, click here to try again.
Discard post?
You will lose what you have written so far.
Personal information found

We found the following personal information in your message:

This information will be visible to anyone who visits or subscribes to notifications for this post. Are you sure you want to continue?

A problem occurred. Please try again.
Create Reply
Edit Reply
Delete post?
This will remove the reply from the Answers section.
Notifications are off
Your notifications are currently off and you won't receive subscription updates. To turn them on, go to Notifications preferences on your Profile page.
Report abuse
Google takes abuse of its services very seriously. We're committed to dealing with such abuse according to the laws in your country of residence. When you submit a report, we'll investigate it and take the appropriate action. We'll get back to you only if we require additional details or have more information to share.

Go to the Legal Help page to request content changes for legal reasons.

Reported post for abuse
Unable to send report.
Report post
What type of post are you reporting?
Google takes abuse of its services very seriously. We're committed to dealing with such abuse according to the laws in your country of residence. When you submit a report, we'll investigate it and take the appropriate action. We'll get back to you only if we require additional details or have more information to share.

Go to the Legal Help page to request content changes for legal reasons.

Reported post for abuse
Unable to send report.
This reply is no longer available.
/machinelearningeducation/threads
//accounts.google.com/ServiceLogin
You'll receive email notifications for new posts at
Unable to delete question.
Unable to update vote.
Unable to update subscription.
You have been unsubscribed
Deleted
Unable to delete reply.
Removed from Answers
Marked as Recommended Answer
Removed recommendation
Undo
Unable to update reply.
Unable to update vote.
Thank you. Your response was recorded.
Unable to undo vote.
Thank you. This reply will now display in the answers section.
Link copied
Locked
Unlocked
Unable to lock
Unable to unlock
Pinned
Unpinned
Unable to pin
Unable to unpin
Marked
Unmarked
Unable to mark
Reported as off topic
/machinelearningeducation/profile/0?hl=en