Add commentary to July

financials/2021-07/breakdown.txt

@@ -1,8 +1,8 @@
-Riju :: $175.43
+Riju :: $169.46
   CloudWatch :: $34.80
-  EC2 :: $112.98
+  EC2 :: $107.01
     Data Transfer :: $0.68
-    EBS Snapshot :: $11.42
+    EBS Snapshot :: $5.45
     EBS Volume :: $46.40
       EBS Volume :: $46.40
         gp2 :: $11.61
@@ -21,3 +21,41 @@ Riju :: $175.43
     LCUs :: $0.06
     Load Balancer :: $19.68
   S3 :: $0.29
+
+COMMENTARY: This month was a disaster because AWS makes it really hard
+to understand what exactly is going to run up your bill.
+
+The most egregious thing here is CloudWatch. It turns out that if you
+follow the official documentation for how to set up a CloudWatch alarm
+on disk space for your EC2 instance, the default configuration has SSM
+Agent creating a metric for *every* filesystem mounted on your
+instance, which is actually one or more per Docker container, so I
+actually had like multiple tens of thousands of metrics being shipped
+to CloudWatch, which is expensive. I fixed this for August, bringing
+CloudWatch costs to be effectively zero.
+
+We have some charges for a t3.medium, this is before I scaled the
+server down to t3.small. The charges for that instance are also higher
+than you'd expect because I was originally running two of them before
+scaling it down for a singleton because I realized I was out of my
+depth.
+
+We had a couple gp2 volumes (more expensive) before I migrated
+everything to gp3. EBS costs are generally quite high here because not
+only did I previously have two instances serving traffic, but I also
+had a dev server. Each of those three instances had to have the full
+256 GB data volume to store language images, which was ridiculously
+expensive. I'm planning on keeping Riju as a singleton for a while
+because of this issue, and relying on vertical scaling until that
+becomes no longer feasible. The persistent dev server will be replaced
+by a transient CI instance that can be spun up to do large rebuild
+operations, mitigating EBS costs.
+
+t3.2xlarge is the dev server, this is mostly just tough luck since I
+did need to spend a lot of time building and rebuilding language
+images and those hours add up. Hopefully that won't be as much of an
+issue going forward now that the infrastructure is more stable and we
+can hopefully get away without a dev server in general. But
+fundamentally you can't do builds on your local laptop without a
+symmetric Internet plan because you need to upload like 100 GB for a
+full rebuild.

financials/fin.py

@@ -149,8 +149,11 @@ def classify_line_item(item, billing_month=None, full=False):
             service == "AmazonEC2"
             and resource != "i-077884b74aba86bac"
             and "ElasticIP:IdleAddress" not in usage_type
+            and "EBS:SnapshotUsage" not in usage_type
         ):
             project = "Riju"
+    # AWS does not let you put tags on a public ECR repository,
+    # yippee.
     if service == "AmazonECRPublic" and resource.endswith("repository/riju"):
         project = "Riju"
     category = [
@@ -287,17 +290,19 @@ def main():
     parser = argparse.ArgumentParser()
     parser.add_argument("date")
     parser.add_argument("-f", "--force-download", action="store_true")
+    parser.add_argument("-w", "--write", action="store_true")
     args = parser.parse_args()
     year, month = map(int, args.date.split("-"))
     billing_month = f"{year}-{month:02d}"
     csv_path = get_csv(year, month, force_download=args.force_download)
     taxonomy = classify_costs(csv_path, billing_month=billing_month)
     print_taxonomy(taxonomy)
-    riju_taxonomy = taxonomy["categories"]["AWS"]
-    riju_taxonomy["categories"] = {"Riju": riju_taxonomy["categories"]["Riju"]}
-    target_dir = ROOT / f"{year}-{month:02d}"
-    with open(target_dir / "breakdown.txt", "w") as f:
-        print_taxonomy(riju_taxonomy, file=f)
+    if args.write:
+        riju_taxonomy = taxonomy["categories"]["AWS"]
+        riju_taxonomy["categories"] = {"Riju": riju_taxonomy["categories"]["Riju"]}
+        target_dir = ROOT / f"{year}-{month:02d}"
+        with open(target_dir / "breakdown.txt", "w") as f:
+            print_taxonomy(riju_taxonomy, file=f)
 
 
 if __name__ == "__main__":