11,262 questions
0
votes
0
answers
45
views
Cloud Build no longer authenticates gsutil using service account
I have a Dockerfile that builds an image based on golang:bookworm. It installs the google-cloud-cli package from the https://packages.cloud.google.com/apt repository. I used it to build an image that'...
- 218
Advice
0
votes
1
replies
70
views
Speeding up unzipping of file in Google Cloud Storage (GCS)
I receive ZIP files everyday to a bucket. Part of my Python pipeline is to extract these out into the individual CSVs. However, wondering if there's a quicker way?
There's roughly 20 files in each ZIP,...
- 5
Advice
0
votes
0
replies
35
views
Does changing the storage class of a GCS object resets its age counter?
I am setting up lifecycle policies for some of my buckets via terraform.
I have this strategy:
standard > nearline > deletion.
The thing is, I am not sure if the AGE property of a file in the ...
- 460
Best practices
0
votes
0
replies
59
views
Optimal way to upload large files (up to 10GB) in GCS from frontend client
My use case is simple in nature.
I have a platform where users can upload any files up to 20GB.
My current solution is:
Frontend Client asks for a presignedURL which the Backend generates
return ...
- 53
1
vote
0
answers
43
views
Is it possible to do offline upload using GCS Bucket Presigned URL resumable vs write
I have a Frontend Client which lets users uploads any number of files of any size (think up to 100 GB File).
Currently I am using GCS buckets presigned URL to upload the file. My current ...
- 53
0
votes
0
answers
31
views
Firebase CLI fails with "An unexpected error has occurred" only when deploying storage
I'm trying to deploy Cloud Storage CORS settings using the Firebase CLI, but it consistently fails with a generic "Error: An unexpected error has occurred."
Project ID:
oa-maintenance-v2
...
- 1
0
votes
0
answers
41
views
Compute Engine image creation fails with “Required ‘read’ permission for storage object of a bucket” from Java SDK, works via REST API and Console
I am trying to create a virtual image from a tar file that is present in the storage bucket
When tried from my Java SDK code which is my actual requirement
I get this
Required ‘read’ permission for ‘${...
2
votes
1
answer
89
views
How to save Google Cloud GET Object API mediaLink response to local storage?
I wrote this code that uses Google's Cloud API to get an object from my bucket and download it. It works perfectly when I had my bucket set to public (allUsers added to Principal w/ all the required ...
- 31
0
votes
1
answer
74
views
distcp creating file in GCP bucket instead of file inside directory
Context:
using distcp, I am trying to copy HDFS directory including files to GCP bucket.
I am using
hadoop distcp -Dhadoop.security.credential.provider.path=jceks://$JCEKS_FILE hdfs://nameservice1/...
- 49
1
vote
2
answers
102
views
ModuleNotFoundError: No module named 'google' in WSL
I was trying to use google cloud storage in a python virtual environment. I tried installing google-cloud-storage and whenever I run the code I always get the error ModuleNotFoundError: No module ...
- 52
0
votes
1
answer
94
views
GCSToGCSOperator is moving folder along with files when using option move_object as True
I have a requirement to move files from a source folder to destination folder in different GCS buckets. I am using GCSToGCSOperator with following config:
source_bucket: "source_bucket"
...
- 1,936
1
vote
0
answers
300
views
Running huggingface models offline using huggingface snapshot download causes errors
I have been trying to run some models from huggingface locally.
The script is being hosted on google cloud run.
Since running the instance multiple times triggers rate limiting, I have downloaded the ...
- 13
0
votes
0
answers
49
views
Firebase storage image urls keep returning access denied response
I have a small app where I am using firebase functions to upload an image into firebase storage.
Once done, I store this image url against an object in firebase db and then reuse this image in the app ...
- 1
1
vote
0
answers
108
views
How can I authenticate and upload files to Google Cloud Storage in Ballerina without manually generating access tokens?
I'm trying to upload files to a Google Cloud Storage bucket from a Ballerina application.
Right now, the only way I’ve found to authenticate is by manually generating an access token using a service-...
0
votes
1
answer
85
views
How to prevent Django from generating migrations when using dynamic GoogleCloudStorage in a FileField?
On my Django Project I have a model that a property is used to store videos in a specific and single Google Cloud Storage bucket, using FileField.
The model is defined like this:
from storages....
- 868