20 Matching Annotations
  1. May 2020
    1. Your Amazon Athena query performance improves if you convert your data into open source columnar formats, such as Apache Parquet

      s3 perfomance use columnar formats

    1. Expedited retrieval allows you to quickly access your data when you need to have almost immediate access to your information. This retrieval type can be used for archives up to 250MB. Expedited retrieval usually completes within 1 and 5 minutes.

      https://aws.amazon.com/glacier/faqs/

      3 types of retrieval

      expecited 1~5minutes

    1. In general, bucket owners pay for all Amazon S3 storage and data transfer costs associated with their bucket. A bucket owner, however, can configure a bucket to be a Requester Pays bucket. With Requester Pays buckets, the requester instead of the bucket owner pays the cost of the request and the data download from the bucket. The bucket owner always pays the cost of storing data.

      Request Pays

    1. Amazon S3 event notifications are designed to be delivered at least once. Typically, event notifications are delivered in seconds but can sometimes take a minute or longer.

      event notification of s3 might take minutes

      BTW,

      cloud watch does not support s3, but cloud trail does

  2. Aug 2019
  3. Aug 2018
  4. Feb 2018
  5. Dec 2017
  6. Oct 2017
  7. Jun 2017
    1. Kafka consumer offset management protocol to keep track of what’s been uploaded to S3

      consumers keep track of what's written and where it left off by looking at kafka consumer offsets rather than checking S3 since S3 is an eventually consistent system.

    2. Data lost or corrupted at this stage isn’t recoverable so the greatest design objective for Secor is data integrity.

      data loss in S3 is being mitigated.

  8. May 2017
  9. Mar 2017
  10. May 2016
    1. . Is art about making up new things or about transforming the raw material that's out there? Cutting, pasting, sampling, remixing and mashing up have become mainstream modes of cultural expression, and fan fiction is part of that. It challenges just about everything we thought we knew about art and creativity.

      Not really. Art has always been about reacting (in some part) to the works that have come before it.