Sign up

Karn Wong

Not verified No WebSub updates No webmention support Not yet validated

Homepage and blog by Karn Wong

Generator
Zola
Public lists
I ♥ RSS
Fetched

Karn Wong
• Unknown

2025-01-26-1

With pixi I can finally remove pyenv from my setup.

Previously needed it to use mamba for installing qgis on ARM mac for geoparquet support

Which means you only need two lines to install qgis via conda:

curl -fsSL pixi.sh/install.sh | bash
pixi global install qgis

2025 is awesome

Karn Wong
• Unknown

2025-01-26-2

Very impressed with flox - dev venv standing on nix ecosystem.

Especially with the fact that it auto detects your project language & runtime means you do not have to setup the flox env from scratch, yay devx.

ถ้าคุณเป็นคนที่ประสาทจะกินกับการเซ็ตคอมของเดฟทุกคนในทีม

หรือ ตบตีปัญหา dev env ที่ไม่เคยตรงกันซักเครื่อง

แต่จะไปไล่เซ็ตทุกอย่างเป็น setup script ก็ไม่ได้ว่างขนาดนั้น

อยากใช้ nix แต่เซ็ตอัพไม่ง่ายสำหรับน้องๆ ในทีม

ขอแนะนำ flox - รมณ์ประมาณ nix แบบ venv

เจ๋งอย่าง คือ flox auto detect language + runtime จาก source code แล้วปั้น flox config มาให้ในระดับนึงแล้ว ชีวิตจะได้สบาย

Karn Wong
• Unknown

My journey for fully switching from Windows over to Linux

Note: This is not a post to bash on Windows. It's more of what I need for work and tinkering stuff are better done on Linux Background My first OS was Windows. My first-ever automation script was written in Batch script, and I learned tons from tinkering with the systems wi...

Karn Wong
• Unknown

2025-01-23

A few weeks back I was optimizing my website to reduce total size. Turns out google analytics took up 300 KB alone. Yeet it out and my homepage is reduced to 130 KB. Yay optimization!

Karn Wong
• Unknown

Information gathering infrastructure

flowchart LR Feed --> Miniflux %% feeds subgraph Homelab Miniflux --> Wallabag end Wallabag -->KOReader %% newsletters subgraph SaaS Newsletters --> Email end subgraph Automation Email --> EmailToEpubScript end EmailToEpubScript --> Syncthing %% de...

Karn Wong
• Unknown

2024-12-27-1

Migrated my site from hugo to zola this week, gotta say using python to convert frontmatter to toml is kinda cheating.

Karn Wong
• Unknown

2024-12-27-2

Migrated authentik to pi4, because it should be in a landing zone. On initial access it took around 5 seconds to load, oof.

Karn Wong
• Unknown

2024-12-25

Installed Ubuntu on Mac Mini 2018 only to find out the WiFi driver is not supported. Thankfully there's a whole wiki on how to install wireless drivers on T2 mac devices. It's a warm and fuzzy feeling to know that the community is always there for your tech ailments.

Karn Wong
• Unknown

2024-12-24

Was browsing HN and saw a thread about Proxmox shredding SSDs. Looking under the console and yup, it's 13% degraded. It's a used Mac Mini 2018 and only been used as Proxmox for a year.

I still love Proxmox, but I only use it to spin up scratch VMs. Maybe I'll use the Mac Mini for distro hopping 😝

Karn Wong
• Unknown

Native implementation vs WASM for Go, Python and Rust benchmark

In 2024 you would hear the term wasm pop up now and then. It is not exactly new, but it's gaining more traction due to wider ecosystem in recent years. Wasm stands for web assembly, essentially it's a format that allows you to execute binaries in browser runtimes. Imagine cr...

Karn Wong
• Unknown

Various notes on career-related stuff

Compensation Sometimes you need to take a pay cut in order to learn new skills. But it will pay off in the end. On the other hand, if you are paid above the market rate, when you have to move it's probably harder because most orgs won't offer the same compensation. Ski...

Karn Wong
• Unknown

Calling C from Go, Python and Rust benchmark

There's a consensus that generally, c is very fast, and python is very slow. But if we are talking about go and rust, you would find that rust is slightly faster than go. So from fastest to slowest: c, rust, go, python. But what if you have go, python and rust calling c func...

Karn Wong
• Unknown

LLM serving latency benchmark

After the rise of ChatGPT in 2023, these days most people are familiar with the concept of LLM - Large Language Model. If you are LLM users, things are looking bright because there are gazillion options for you to choose from. But if you are on the other side - creating and ...

Karn Wong
• Unknown

AWS IAM credentials best practices

It's hard to escape AWS, seeing how prevalent it is in global internet infrastructure. Chances are, most websites you visit are hosted on AWS. As a software engineer, you probably encounter AWS at certain point in your career, and while getting AWS IAM credentials to work lo...

Karn Wong
• Unknown

When (not) to use Terraform

If we are talking about IaC, Terraform would be on the list. It made IaC popular and help a lot of companies maintain infrastructure at scale. Especially when you have multiple sets of infrastructures to maintain, Terraform can help you reduce the setup time tremendously via...

Karn Wong
• Unknown

Hello World API Performance Benchmark (Go, Node, Python, Rust)

Correction 2024-09-21: after using multi-stage build, Node image size dropped from 1.1GB to 130MB. The first programming language I achieved proficiency was Python, so for the longest time I've been using it to do most stuff. Last year I picked up Go, and I had a blast with...

Karn Wong
• Unknown

Simplify self-hosting backups to S3 with docker

These days there are multiple ways to deploy a workload, be it cloud-based or bare-metal. For cloud, depending on whether you are using PaaS or IaaS, backup options can vary. Why do we need to backup? Because your workloads can contain a state, this can be stored as local fi...

Karn Wong
• Unknown

Streamlit load test performance

Streamlit is well-loved by many people, especially among data folks due to the fact that it does not require prior web programming knowledge to get started. Popular use cases for streamlit can be anything from a quick machine learning application poc or internal dashboards. ...

Karn Wong
• Unknown

Reasons why you shouldn't use programming languages for IaC

When it comes to IaC (infrastructure as code), most people might have heard of HashiCorp's Terraform (it uses HCL as DSL. Interestingly enough, Terraform also has its own CDK to translate programming languages into HCL), Pulumi or AWS CDK. The latter two support programming ...

Karn Wong
• Unknown

GCP's service account credentials can be a security risk. Here's how to mitigate them.

If you look online, many sources would tell you that you should use service account to authenticate for GCP services. While this is true, it's not for all the cases. For local development, you should use Application Default Credentials Imagine working in a team, and you hav...

Karn Wong
• Unknown

Thoughts on summarization service system design

For a summarization task, there should be an input, in which it's reduced to a handful of paragraphs. This input is in text format. You don't necessarily start from a text format though, since the source content can be audio or video files. But this means at the end, the sou...

Karn Wong
• Unknown

Faster spark workloads with Comet

For big data processing, spark is still king. Over the years, many improvements have been made to improve spark performance. Databricks themselves created photon, a spark engine that can accelerate spark queries, but this is proprietary to Databricks. Other alternatives do e...

Karn Wong
• Unknown

Slim down Python Docker image size with Poetry and Pip

Python package management is not straightforward, seeing default package manager (pip) does not behave like node's npm, in a sense that it doesn't track dependencies versions. This is why you should use poetry to manage python packages, since it creates a lock file, so you c...

Karn Wong
• Unknown

Dataframe write performance to Postgres

Previously, I talked about dataframe performance, but this doesn't include writing data to destination part. At a large scale, big data means you need to use spark for data processing (unless you prefer SQL, in which this post is irrelevant). But not many orgs need big data,...

Karn Wong
• Unknown

How to connect to Cloud SQL from Cloud Run (no, you don't need a VPC)

A minimal application architecture would compose of a database, and an application backend. Serverless database is still in its infancy, but thankfully container-based runtime is very much alive and doing well. On GCP, a serverless container-based runtime do exist, known as ...

Karn Wong
• Unknown

What is platform engineering?

Back in 2017-2018, everyone wanted to be a data scientist. Then reality hits, that they need a data engineer for a successful machine learning project. Things didn't end there, since they also need a machine learning engineer to create production-ready code. Some people thin...

Karn Wong
• Unknown

Collaboration model for data science projects

Many data science teams are struggling with implementing end-to-end machine learning projects. It's a very common phenomenon, so if you are experiencing this, you are not alone. Having worked in every stage of data science project lifecycle, in addition to normal web service...

Karn Wong
• Unknown

Should data scientists deploy models to production?

Over the years I've heard stories of data teams struggling with deploying machine learning models to production. Clearly there is a pattern here. This article is my reflection on the matter. So what's the problem? Data scientists, by definition, create mathematical models f...

Karn Wong
• Unknown

Setting up Postgres locally, what could go wrong?

There are multiple reasons why someone wants to set up a postgres locally. Either for learning SQL or as an application's backend. Over the years I see people struggle with using postgres locally, so here are common use cases and possible issues, with solutions for each. For...

Karn Wong
• Unknown

Things to watch out for GCP SSL with Cloudflare DNS

For our production workload, we deploy the workloads on Kubernetes, in which an ingress resource is created per each deployment. Resources in ingress are GCP Load Balancer and SSL Certificate. As for DNS, we use Cloudflare since it enables CDN without extra configurations on our part.

A few months after the deployment went live initially, we were informed that the website couldn't be accessed. Turns out GCP couldn't renew the SSL Certificate (error FAILED_NOT_VISIBLE.) Looking at GCP docs, turns out if the DNS couldn't be resolved to the Load Balancer IP, it couldn't provision/renew a certificate.

The fix? Disable proxy on Cloudflare records and wait until GCP successfully renews the certs.