Migrating to a different storage provider is made easier because they all offer the same services and APIs. Users aren’t locked into providers because they rely on a particular feature of the provider. Also, files are content-addressed, enabling them to be transferred directly between miners without the user having to download and re-upload the files. Traditional cloud storage providers lock users by making it cheap to store files but expensive to retrieve them again. Filecoin avoids this by facilitating a retrieval market where miners compete to give users their files back as fast as possible, at the lowest possible price.
Many who participate in the IETF are most comfortable making what we believe to be purely technical decisions; our process is defined to favor technical merit, through our well-known mantra of "rough consensus and running code."
Nevertheless, the running code that results from our process (when things work well) inevitably has an impact beyond technical considerations, because the underlying decisions afford some uses while discouraging others. While we believe we are making only technical decisions, in reality, we are defining (in some degree) what is possible on the Internet itself.
This impact has become significant. As the Internet increasingly mediates essential functions in societies, it has unavoidably become profoundly political. it has helped people overthrow governments and revolutionize social orders, swing elections, control populations, collect data about individuals, and reveal secrets. It has created wealth for some individuals and companies while destroying others'.
Compatibility with the S3 API has become the baseline upon which companies add data analytics features.
- data lakes for streaming data, such as system events and logs, application telemetry, sensor readings, financial transactions, online interactions, social media activity and other metadata;
- object storage databases for metadata, unstructured content and binary large objects;
- storage for big data analytics using software like Spark, Flink, Hive and their commercial alternatives;
- machine and deep learning training data and input streams for analytics using previously trained models;
search engine repositories;
- rich media streaming;
- persistent data stores for container-based and cloud-native applications;
- and repositories for software development environments, including source code management, continuous integration and continuous delivery pipelines, issue tracking and documentation
How does object storage and s3 compatibility fit in the Filecoin vision?
Why would object storage with Filecoin be more desirable than what exists today?
What would it take to move the largest dapps off of AWS?
- what do they use it for?
- how much data lives there for now?
- how much data could live on IPFS instead?
why is object storage more scalable?
"An object storage system works well here since the data is static or fixed. In this example, the contents of the book will not change. The objects (data, metadata, and ID) are stored as “packages” in a flat structure and easily located and retrieved with a single API call."
is the bucket the object in object storage?
no. object refers to the way data is grouped as a unit as opposed to being spread out across folders in a file system, or unstructured in a data lake.
Files, blocks, and objects are storage formats that hold, organize, and present data in different ways—each with their own capabilities and limitations. File storage organizes and represents data as a hierarchy of files in folders; block storage chunks data into arbitrarily organized, evenly sized volumes; and object storage manages data and links it to associated metadata.
Think of a closet full of file cabinets. Every document is arranged in some type of logical hierarchy—by cabinet, by drawer, by folder, then by piece of paper. This is where the term hierarchical storage comes from, and this is file storage. It is the oldest and most widely used data storage system for direct and network-attached storage systems, and it’s one that you’ve probably been using for decades. Any time you access documents saved in files on your personal computer, you use file storage. File storage has broad capabilities and can store just about anything. It’s great for storing an array of complex files and is fairly fast for users to navigate.