Learnings

An optionated collection of (mostly) technical things every software developer should know

the problems with trusting models too much, especially with not enough data

s3fs is an awesome Python library that makes working with AWS S3 as easy as working with local files:

Needed to get the row count of a huge CSV file in S3 without downloading it. It’s possible by using S3 Select functionality! It makes CSV and JSON files queryable with a SQL-like syntax. See: #S3.Client.select_object_content

Needed to concatenate several CSV files in S3. Downloading them would be costly (both on time and money cost). There are scripts that do it without downloading, by using “multi-part upload“ from S3 to S3. See: #s3fs.core.S3FileSystem.merge and

How to Combine Two Python Decorators:

The Product-Minded Software Engineer:

Excel Viewer: extension for VSCode that provides read-only previews for CSV files and Excel spreadsheets. The contents of the file will be displayed in a FlexGrid control, which supports sorting and filtering via its column headers

A framework for product strategy

Microsoft has an intelligent end-to-end analytics for safe deployment in large-scale cloud named Gandalf:

It's possible to reset all variables in an iPython shell without having to restart it by calling %reset -f #magic-reset

What are coachmarks, and how they can improve the user experience

Fun Vim shortcut of the day: map <F7> :!pytest %<CR> "Run pytest for the current file on buffer

Identify your slowest test set using pytest by using the flag --duration, e.g. --duration=50 will report the 50 slowest test set from a given test suite. Refer to: #profiling-test-execution-duration

Train sklearn 100x faster:

Fundamentals of color in interface design

If you're not using Oh My Zsh you totally should! Your shell will get VERY powerful.

Django's GZipMiddleware compresses content for browsers that understand GZip compression #module-django.middleware.gzip

Fix path to import local modules in Airflow DAG code:

Dynamically Generating DAGs in Airflow:

"Once at a certain level, all problems are solved by people. There is no such thing as purely technical problems" says Silvia Botros on principal engineering:

Easy to understand explanation of Big-O notation without any fancy terms:

BLUF: a strategy for effective communication

pdb.pm() puts a breakpoint at the traceback found in sys.last_traceback . This is really useful when you're running some method in a console and you need a breakpoint at the moment the code threw the exception. See more at: #pdb.pm