Jeff Triplett's Micro.blog

About Archive RSS

  • Django

    ,

    Python

    ,

    LLM

    🤖 I released files-to-claude-xml and new development workflows

    After months of using and sharing this tool via a private gist, I finally carved out some time to release files-to-claude-xml.

    Despite my social media timeline declaring LLMs dead earlier today, I have used Claude Projects and Artifacts.

    My workflow is to copy a few files into a Claude Project and then create a new chat thread where Claude will help me write tests or build out a few features.

    My files-to-claude-xml script grew out of some research I did where I stumbled on their Essential tips for long context prompts which documents how to get around some file upload limits which encourages uploading one big file using Claude’s XML-like format.

    With files-to-claude-xml, I build a list of files that I want to import into a Claude Project. Then, I run it to generate a _claude.xml file, which I drag into Claude. I create a new conversation thread per feature, then copy the finished artifacts out of Claude once my feature or thread is complete.

    After the feature is complete, I delete the _claude.xml file from my project and replace it with an updated copy after I re-run files-to-claude-xml.

    Features on the go

    One bonus of using Claude Projects is that once everything is uploaded, I can use the Claude iOS app as a sort-of notes app and development tool. I can start parallel conversation threads and have it work on new ideas and features. Once I get back to my desktop, I can pull these chat conversations up, and if I like the direction of the feature, I might use them. If not, I have wasted no time or effort on them. This also serves as a nice ToDo list.

    New workflows

    I am working on side projects further using this methodology. Sometimes, I would like to work on something casually while watching Netflix, but my brain shuts off from coding during the day. Instead of feeling bad that I haven’t added share links to a website or some feature I meant to add last week, I can pair Claude to work on it with me.

    I can also get more done with my lunch hours on projects like DjangoTV than I could have otherwise. Overall, I’m happy to have an on-demand assistant to pair with and work on new features and ideas.

    It’s also quicker to try out new ideas and projects that I would have needed to make time for.

    Alternatives

    Simon Willison wrote files-to-prompt, which I think is also worth trying. I contributed to the discussion, feedback, and document structure for the --cxml feature.

    I wrote files-to-claude-xml before Simon had cxml support and hoped to not release my version.

    However, after trying it out on several projects, my ignore/exclude list grew more significant than the files that I wanted to include in my project to send to Claude. I found it easier to generate a list of files to pass to mine instead of maintaining a long list to exclude.

    Saturday October 12, 2024

  • Django

    ,

    Python

    ,

    UV

    ⚙️ UV with GitHub Actions to run an RSS to README project

    For my personal GitHub profile, I list my activities, affiliations, and the latest updates from some of my projects.

    Historically, I have used JasonEtco/rss-to-readme GitHub Action to fetch a few RSS feeds or two and to update my README a few times a day.

    Overall, I’m happy with this setup. I used it on the Django News GitHub Organization to pull in newsletter issues, jobs, and the latest videos from our various projects. When I tried to install rss-to-readme in our repo, I was getting node12 errors. (Have I mentioned how much I loathe node/npm?).

    Instead of forking rss-to-readme and trying to figure out how to upgrade it, I used this as an excuse to “pair program” with Claude. We quickly built out a prototype using Python and the feedparser library.

    I would share the chat log, but it’s mostly me trying out a few different ways to invoke it before I settle on the finished approach. See the source code over on GitHub if you are curious: https://github.com/django-news/.github/blob/main/fetch-rss.py

    Once I had a working Python script that could fetch an RSS file and modify the README, I decided to run/deploy it using UV to see how minimal I could build out the GitHub Action.

    GitHub Action

    To run our fetch-rss.py script, we have four steps:

    1. actions/checkout Get a git checkout of our project.
    2. astral-sh/setup-uv Setup UV also installs Pythons for us. As a bonus, we enabled UV’s cache support, which will run much faster in the future unless we change something in our fetch-rss.py file.
    3. Run uv run fetch-rss.py ... to fetch our RSS feeds and write them to disk. uv run installs any dependencies and caches them before our fetch-rss.py runs.
    4. stefanzweifel/git-auto-commit-action If our README.md file has changed, save our changes and commit them back to git and into our README.

    Our schedule.yml GitHub Action workflow runs twice daily or whenever we push a new change to our repo. We also set workflow_dispatch, which gives us a button to run the script manually.

    # .github/workflows/schedule.yml
name: Update README

on:
  push:
    branches:
      - main
  schedule:
    # Once a day at 12 AM
    - cron: 0 12 * * *
  workflow_dispatch:

jobs:
  update:
    runs-on: ubuntu-latest

    permissions:
      contents: write

    steps:
      - uses: actions/checkout@v4

      - name: Install uv
        uses: astral-sh/setup-uv@v3
        with:
          enable-cache: true
          cache-dependency-glob: |
                        *.py

      - name: Fetch our Feeds
        run: |
          # Fetch latest Django News Newsletter entries
          uv run fetch-rss.py \
              --section=news \
              --readme-path=profile/README.md \
              https://django-news.com/issues.rss          

      - uses: stefanzweifel/git-auto-commit-action@v5
        with:
          commit_message: ":pencil: Updates README"

    Results

    Overall, I’m pleased with this solution. If I wanted to spend more time on it or re-use this workflow, I might turn it into a GitHub Action workflow so that we can call: django-news/rss-to-readme to use in other projects. For now, this is fine.

    I’m happy with the astral-sh/setup-uv and uv run steps because they save me from having to set up Python and then install our project dependencies as separate steps.

    I normally shy away from running Python workflows like this in GitHub Actions because they involve a lot of slow steps. This entire workflow takes 16 to 20 seconds to run, which feels fast to me.

    Saturday October 5, 2024

  • Django

    ,

    Python

    🎉 Announcing DjangoTV

    Friends, I’m launching a half-finished website idea that I have been playing around with for a while. djangotv.com is focused on promoting and searching Django videos to make discovery easier.

    I wanted to launch DjangoTV before I left DjangoCon US 2024, so I’m technically announcing it from the airport. Last year, I launched Django News Jobs during the Django sprints because I was annoyed by the state of the various Django job boards.

    After a year of hearing people complain about Django and Python YouTube videos not getting enough views, I decided to address the problem by building out a website to help organize and promote them.

    DjangoTV is not competing with pyvideo.org. PyVideo is a fantastic resource for the Python community, and one of my goals is to make it easier to get Django content on PyVideo, too.

    DjangoTV is incomplete, and I have many ideas to improve it, including backfilling some of our older conferences, adding presenters, and adding some other metadata to make it a useful community resource.

    I focused on DjangoCon videos to bootstrap the project, and we’ll slowly expand our archives. I have been extracting good quotes, adding transcriptions, and adding better summaries. Please expect this to change a lot. I even launched the website with DEBUG turned on for a few days before I turned it off.

    Your feedback is crucial to us. If you have any ideas or suggestions, please don’t hesitate to share them.

    Most importantly, names change, people change, and bodies change. It’s essential to me that we respect each other and handle this with care. If you have a concern or need us to pull a video, please don’t hesitate to reach out; we will take care of it.

    Friday September 27, 2024

  • Django

    ,

    Python

    ,

    Today I Learned

    🧳 DjangoCon US, Black Python Devs Leadership Summit, and Django Girls Durham

    I’m heading to Durham, NC, for seven days of DjangoCon US this Friday. This is my 10th year volunteering and the 9th year that DEFNA, the non-profit I co-founded, has run a DjangoCon US event. Here is an overview of the week.

    Black Python Devs Leadership Summit (Saturday)

    I’m attending and speaking on a discussion panel on Saturday at the Black Python Devs Leadership Summit. Tickets are free, and they will be streaming online in the afternoon. Donations are accepted and appreciated.

    Django Girls Durham (Saturday)

    Django Girls are hosting a Django workshop and teaching beginners a crash course on building their first website using Django.

    DjangoCon US Tutorials (Sunday)

    On Sunday morning, I’ll be volunteering and helping out at the tutorials. In the afternoon, we have a tradition of stuffing swag bags, which takes a big group and is a fun way to kick off the conference. You do not need a tutorial ticket or an organizer to help out. Ask at the registration desk, and they can direct you to when and where we are doing this.

    Django Social meetup (Sunday)

    My company REVSYS is sponsoring a DjangoSocial Raleigh/Durham Pre-DjangoCon Special meetup on Sunday evening before the conference kicks off. The meetup will be great for meeting other attendees the night before the conference.

    DjangoCon US Talks (Monday through Wednesday)

    The talks are great, but the busiest three days of the conference are also the busiest. There is always a lot going on, from sun up to sun down.

    DjangoCon US Sprints (Thursday and Friday)

    The sprints are one of my favorite parts of the conference. In past years, I have been so exhausted by the sprints that it’s hard to sit down and focus. It’s one of the best times to discuss Django and the Django ecosystem. If you have a project or want to find a project to help with, the sprints are great for getting your feet wet.

    Outro

    Tickets are still available if you live near Durham and want to attend. Both events have online and in-person options, so there is no pressure to make last-minute travel plans.

    If you live around Durham and want to meet up, please reach out. Let’s see if we can meet for coffee.

    Friday September 20, 2024

  • Django

    ,

    Python

    ,

    UV

    ,

    Today I Learned

    🤠 UV Roundup: Five good articles and a pre-commit tip

    I have written quite a bit about UV on my micro blog, and I am happy to see more and more people adopt it. I have stumbled on so many good articles recently that I wanted to share them because every article points out something new or different about why UV works well for them.

    If you are new to UV, it’s a new tool written by Astral, the creators of Ruff.

    I like UV because it replaces, combines, or complements a bunch of Python tools into one tool and user developer experience without forcing a UV way of doing it. UV effectively solves the question, “Why do I need another Python tool?” to do everyday Python tasks.

    Some reason I like UV after using it for months:

    • It’s a faster pip and is really, really fast
    • It can install and manage Python versions
    • It can run and install Python scripts
    • It can run single-file Python scripts along with their dependencies
    • It can handle project lock files

    While some people don’t care about UV being fast, it’s shaved minutes off my CI builds and container rebuilds, which means it has also saved me money and energy resources.

    Overall thoughts on UV

    Oliver Andrich’s UV — I am (somewhat) sold takes the approach of only using UV to set up a new Python environment. Oliver uses UV to install Python, aliases to call Python, and UV tool install to set up a few global utilities.

    Using UV with Django

    Anže Pečar’s UV with Django shows how to use UV to set up a new project with Django.

    Switching from pyenv to UV

    Will Guaraldi Kahn-Greene’s Switching from pyenv to uv was relatable for me because I also use pyenv, but I plan to slowly migrate to using only UV. I’m already halfway there, but I will have pyenv for my legacy projects for years because many aren’t worth porting yet.

    Using UV and managing with Ansible

    Adam Johnson’s Python: my new uv setup for development taught me to use uv cache prune to clean up unused cache entries and shows how he manages his UV setup using Ansible.

    Some notes on UV

    Simon Willison’s Notes on UV is an excellent summary of Oliver’s notes.

    A parting UV tip

    If you are a pre-commit fan hoping for a version that supports UV, the pre-commit-uv project does just that. I started updating my justfile recipes to bake just lint to the following uv run command, which speeds up running and installing pre-commit significantly.

    $ uv run --with pre-commit-uv pre-commit run --all-files
pre-commit-uv

    If you are attending DjangoCon US…

    If you are attending DjangoCon US and want to talk UV, Django, Django News, Django Packages, hit me up while you are there.

    I’ll be attending, volunteering, organizing, sponsoring, and sprinting around the venue in Durham, NC, for the next week starting this Friday.

    We still have online and in-person tickets, but not much longer!

    Thursday September 19, 2024

  • Django

    ,

    Python

    ,

    Office Hours

    📅 Office Hours Fall Update

    This Friday, I am hosting Office Hours before I travel to DjangoCon US (organizer) and the Black Python Devs Leadership Summit (speaker) in Durham, NC.

    This Friday will be my last session before a two-week break, but I will resume Office Hours again on October 4th.

    • Friday, September 13th, 2024, 2:30 pm to 4:30 pm
    • Friday, September 20th, 2024 - No office hours
    • Friday, September 27th, 2024 - No office hours
    • Friday, October 4th, 2024, 2:30 pm to 4:30 pm

    High-level details

    ℹ️ Anyone can join office hours. Many join because they work remotely, miss seeing faces, and miss the random conversations when a small group hangs out.

    ℹ️ Our office hours are a collaborative space where we can discuss our ongoing projects, catch up, and work together to wrap up our week on a productive note.

    🙏 As always, everyone is welcome to join, whether you’re a regular attendee or joining for the first time. If you are curious, reach out.

    ✅ If you need any additional details, feel free to send me a message or check out the gist from our previous sessions, where you’ll always find the most recent Zoom link ⚠️

    Thursday September 12, 2024

  • Django

    ,

    Today I Learned

    🚜 On Evolving Django's `auth.User` model

    I need to re-read/re-listen to Carlton’s Evolving Django’s auth.User Stack Report before I have a firm opinion, but I saw his Mastodon post and I thought it was worth sharing some initial thoughts.

    On my first read, I fell into the camp that wants django-unique-user-email and full name to be the defaults on Django’s built-in User. I have wanted this forever, and my clients do, too.

    Django should still allow me to create a custom user, but the documentation could be better. Django’s documentation is a topic for another day, but I’m confused by it and find it less and less helpful. I notice the developers I work with get lost in them, too. Docs may be a good DjangoCon US topic if you want to discuss it in a few weeks.

    I would love the default User to have a unique email address, full name, and a short name. Optionally, a username is an email address setting. Please don’t take away the ability to have a custom User model. (Update: Carlton is not making a case for removing a custom User model, but others have made that argument. No, thank you.)

    Carlton’s “Action points” conclusion seems reasonable to me.

    Tuesday September 10, 2024

  • Django

    ,

    Python

    🚫 Stop scheduling security updates and deprecating major features over holidays

    I know people outside the US 🙄 at this, but please stop releasing major security updates and backward incompatible changes over major US, international, and religious holidays.

    Given that major security updates are embargoed and scheduled weeks and months in advance, it’s essential to coordinate and avoid conflicts. A simple check of the calendar before scheduling announcements can prevent such issues.

    Even if you give everyone two weeks' notice, aka what GitHub just did, wait to schedule them for release over a holiday weekend.

    Historically, the Python and Django communities have also been guilty of this, so I’m not just finger-pointing at GitHub. We can all do better here.

    Update: 100% unrelated to this: Django security releases issued: 5.1.1, 5.0.9, and 4.2.16. Thank you, Natalia (and Sarah) for scheduling this after the US is back from a major holiday.

    Tuesday September 3, 2024

  • Django

    ,

    Python

    ,

    UV

    🚜 Using Claude 3.5 Sonnet to refactor one of Brian Okken's Python projects

    Brian Okken posted and published his Top pytest Plugins script and then a follow-up post, Finding the top pytest plugins, which was pretty cool.

    I have written a few throw-away scripts, which William Vincent wrote about and updated a few times in the Top 10 Django Third-Party Packages (2024) and The 10 Most-Used Django Packages (2024).

    These efforts are powered by Hugo van Kemenade’s excellent Top PyPI Packages.

    This inspired me to fork Brian’s top-pytest-plugins project, which I updated to support passing in other package names like “django” to get a rough estimate of monthly package downloads.

    The refactored project is jefftriplett/top-python-packages.

    Please note: Looking at the package name doesn’t scale as well for projects that have their own Trove classifiers. For a project like pytest, it works well. Many of the top packages may not even have Django in their name for a project like Django. Some projects may even actively discourage a project from using their project in their package’s name for trademark reasons. So, YMMV applies here.

    Prompts

    I added uv run support, which I have written about a lot lately.

    I also copied the top_pytest.py file into a Claude 3.5 Sonnet session, and I let it handle the whole refactor. It even handled adding the PEP 723 new package dependencies without me asking it to.

    In case it’s useful to anyone, here are my prompts:

    ## Prompt:
Please update this script to use a rich table.


## Prompt:
Please update the table styles to be ascii so I can copy and paste it into a markdown doc


## Prompt:
Please remove the description column


## Prompt:
Please change all PyTest and pytest references to Django and django


## Prompt:
Please add back `if 'django' in project.lower() and 'django' != project.lower():`


## Prompt:
please remove the \*# Export to markdown section. I can just pipe the output \*


## Prompt:
Please add the typer library.


## Prompt:
Please remove days and limit


## Prompt:
Please refactor the script to allow me to pass the package name instead of django. You can default to django though.


This way I can pass pytest or flask or other projects.


## Prompt:
Please change the default Table box type to MARKDOWN

    Outro

    I don’t usually write about Claude or prompts, but the tool has been handy lately.

    If you have had some similar successes, let me know. I have been exploring some rabbit holes, and it’s changing the way I approach solving problems.

    Sunday August 25, 2024

  • Django

    ,

    Python

    ,

    UV

    ,

    Today I Learned

    📓 UV Run Django Notes

    I wanted to know how hard it would be to turn one of my django-startproject projects into a uv run friendly project. As it turns out, it worked, and the steps were more than reasonable.

    Before the PEP 723’ing…

    I started with a fairly vanilla manage.py that Django will give you after running python -m manage startproject.

    """Django's command-line utility for administrative tasks."""

import os
import sys


def main():
    """Run administrative tasks."""
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings")
    try:
        from django.core.management import execute_from_command_line
    except ImportError as exc:
        raise ImportError(
            "Couldn't import Django. Are you sure it's installed and "
            "available on your PYTHONPATH environment variable? Did you "
            "forget to activate a virtual environment?"
        ) from exc
    execute_from_command_line(sys.argv)


if __name__ == "__main__":
    main()

    shebang

    Then we add #!/usr/bin/env -S uv run to the top of our manage.py file.

    Next, we make our manage.py executable and try to run it.

    $ chmod +x manage.py
$ ./manage.py
ModuleNotFoundError: No module named 'django'

    Our script ran, but Python couldn’t find Django. To tell our script to install Django, we can use uv add—- script to add it.

    $ uv add --script manage.py django
Updated `manage.py`
$ ./manage.py
...

Type 'manage.py help <subcommand>' for help on a specific subcommand.

Available subcommands:

[django]
    check
    compilemessages
    createcachetable
    dbshell
    diffsettings
    dumpdata
    flush
    inspectdb
    loaddata
    makemessages
    makemigrations
    migrate
    optimizemigration
    runserver
    sendtestemail
    shell
    showmigrations
    sqlflush
    sqlmigrate
    sqlsequencereset
    squashmigrations
    startapp
    startproject
    test
    testserver
Note that only Django core commands are listed as settings are not properly configured (error: No module named 'environs').

    Django worked as expected this time, but Python could not find a few third-party libraries I like to include in my projects.

    To add these, I passed the other four to uv add --script which will add them to the project.

    $ uv add --script manage.py django-click "environs[django]" psycopg2-binary whitenoise
Updated `manage.py`
...
$ ./manage.py
...

    Our Django app’s manage.py works when we run it.

    After the PEP 723’ing…

    After we installed our dependencies in our manage.py file, they were added to the top of the file between the /// blocks.

    #!/usr/bin/env -S uv run
# /// script
# requires-python = ">=3.10"
# dependencies = [
#     "django",
#     "django-click",
#     "environs[django]",
#     "psycopg2-binary",
#     "whitenoise",
# ]
# ///
"""Django's command-line utility for administrative tasks."""

import os
import sys


def main():
    """Run administrative tasks."""
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings")
    try:
        from django.core.management import execute_from_command_line
    except ImportError as exc:
        raise ImportError(
            "Couldn't import Django. Are you sure it's installed and "
            "available on your PYTHONPATH environment variable? Did you "
            "forget to activate a virtual environment?"
        ) from exc
    execute_from_command_line(sys.argv)


if __name__ == "__main__":
    main()
    Friday August 23, 2024

  • Django

    ,

    Python

    ,

    Office Hours

    💼 Office Hours this Friday, August 9th

    I am hosting Office Hours this Friday, August 9th, 2024, from 2:30 pm to 4:30 pm Central Time.

    💼 I plan to upgrade some projects to work with Django 5.1 and possibly Python 3.13 as time permits. I have a few Django Packages tasks to finish too.

    ℹ️ Anyone can join office hours.

    ℹ️ Our office hours are a collaborative space where we can discuss our ongoing projects, catch up, and work together to wrap up our week on a productive note.

    🙏 As always, everyone is welcome to join, whether you’re a regular attendee or joining for the first time.

    ✅ If you need any additional details, feel free to send me a message or check out the gist from our previous sessions, where you’ll find the Zoom link ⚠️

    I look forward to seeing everyone.

    Thursday August 8, 2024

  • Django

    ,

    Python

    ,

    Today I Learned

    ⬆️ Which Django and Python versions should I be using today?

    Django 5.1 was released, and I was reminded of the article I wrote earlier this year about Choosing the Right Python and Django Versions for Your Projects.

    While I encouraged you to wait until the second, third, or even fourth patch release of Django and Python before upgrading, I received a bit of pushback. One interesting perspective claimed that if everyone waits to upgrade, we don’t find critical bugs until the later versions. While that may be plausible, I don’t believe that the dozens of people who read my blog will be swayed by my recommendation to wait for a few patch releases.

    I could have emphasized the potential risks of not testing early. Please start testing during the alpha and release candidate phase so that when Django 5.1 is released, your third-party applications will be ready and working on launch day, minimizing the risk of last-minute issues.

    Today, I tried to upgrade Django Packages to run on Django 5.1 to see if our test suite would run on Django 5.1, and it very quickly failed in CI due to at least one package not supporting 5.1 yet. Even if it had passed, I’m 90% sure another package would have failed because that’s the nature of running a new major Django or Python release on day one. Even if the third-party package is ready, the packaging ecosystem needs time to catch up.

    Which version of Django should I use today?

    I’m sticking with Django 5.0 until Django 5.1’s ecosystem has caught up. I plan to update the third-party packages I help maintain to have Django 5.1 support. After a few patch releases of Django 5.1 have come out and the ecosystem has time to catch up, I will try to migrate again.

    Which version of Python should I use today?

    I’m starting new projects on Python 3.12, with a few legacy projects still being done on Python 3.11. While I am adding Django 5.1 support, I plan to add Python 3.13 support in my testing matrixes to prepare everything for Python 3.13’s release this fall.

    Office hours

    I plan to spend some of my Office Hours this week working on Django 5.1 and Python 3.13 readiness for projects I maintain. Please join me if you have a project to update or would like some light-hearted banter to end your week.

    Wednesday August 7, 2024

  • Django

    ,

    Office Hours

    📅 Office Hours this Friday, July 26th

    I am hosting Office Hours this Friday, July 26th, 2024, from 2:30 pm to 4:30 pm Central Time.

    I’ll probably be on a bit early depending on much of my client projects I get wrapped up this week.

    ℹ️ Anyone can join office hours.

    ℹ️ Our office hours are a collaborative space where we can discuss our ongoing projects, catch up, and work together to wrap up our week on a productive note.

    💼 I will be working on Django Packages and possibly a new project.

    🙏 As always, everyone is welcome to join, whether you’re a regular attendee or joining for the first time.

    ✅ If you need any additional details, feel free to send me a message or check out the gist from our previous sessions, where you’ll find the Zoom link ⚠️

    I look forward to seeing everyone.

    Thursday July 25, 2024

  • Weeknotes

    ,

    Django

    ,

    Python

    ,

    Today I Learned

    📓 Weeknotes for Week 28: July 8 to 14

    I’m running a week behind on this.

    This week was our first week back home without traveling in a month, and it felt good to be home. I had time to catch up on mowing the yard, and I treated the yard with an eco-safe mosquito repellent. Despite the hot weather, Sunday felt nice outside to be mosquito-free.

    I rolled my above-ground sprinkler system A few years ago, and I still need to install and run it this year. I wanted to get it this weekend, and here we are.

    Family

    I converted my daughter’s crib to a daybed over the weekend, and we have been using it for two nights and two naps without any issues. My son took to the board game Risk in Chicago, so I installed the iPad version and walked him through it. It was a pizza and tacos weekend because it was a long week.

    Work

    Occasionally, a project feels like you signed up for a race, but the distance keeps changing whenever you are within sight of the finish line. A project we have been finishing up keeps growing.

    Community Work

    Side projects

    • Django News Newsletter: We shipped issue #241.

    • Django News Jobs: This week, we picked up more jobs that weren’t from one source. I need to write a tool to help maintain this, but it’s a manageable load.

    • I bought a new domain name for a project this weekend. More on that soon.

    Side Quests

    • I dusted off my YouTube-to-Frontmatter tool and added the ability to pull playlists from a given username. I wrote the files out as JSON and used DuckDB to query them, which worked amazingly well.

    • I wrote an Amazon product image downloader for a few blog posts. When the product API did not work, I punted and had ChatGPT write a playwright scraper. It was faster and much less frustrating. I need this for several projects.

    • I cleaned up my sitemaps research tool.

    • I tried out a screenshots-to-code project and ran some 00s-era websites through it that I wish still existed. If someone wants to give me a few years of funding, I think we can make the web not suck again.

    Writing

    2024-07-14: 🔥 Why I deleted Firefox from my machines this weekend I no longer trust or believe in Mozilla, so I deleted Firefox from my machines this weekend. ➜ brew …

    2024-07-13: 🦆 DuckDB may be the tool you didn’t know you were missing 🤔 I haven’t fully figured out DuckDB yet, but it’s worth trying out if you are a Python …

    2024-07-12: 🚜 macOS Bartender app to Ice app I upgraded my Macs to macOS Sonoma a few weeks ago. While everything has been uneventful, the …

    2024-07-11: 🎮 8BitDo Golden/Silver Limited Edition controllers My favorite third-party video game hardware company, 8BitDo, announced its 11th-anniversary limited …

    2024-07-10: 📅 Office Hours for July 12th Office Hours returns this Friday, July 12th, 2024, from 2:30 pm to 4:30 pm Central Time. ℹ️ Anyone …

    2024-07-09: 🔓 Sharing is Caring: How a Simple Sudo Question Led to Better Solutions One of the fun discoveries of blogging is finding your article in search results while trying to …

    2024-07-08: 📓 Weeknotes for Week 27: July 1 to 7 The last week was a blur between the holiday, travel, and cramming a lot of work. My notes this week …

    Entertainment

    📺 Vikings: Valhalla

    📺 The Marvels (2023) - This movie was better than people gave it credit for. It wasn’t my favorite, but it was fun to watch.

    📺 Defending Jacob - I skipped to the end of this series.

    📺 The Last Thing He Told Me - I skipped to the end of this series.

    📺 Presumed Innocent - I surprised myself that I’m still keeping up with this series, but there are only a few weeks left.

    📺 The Acolyte - We are ready for the last episode.

    📺 Atlas (2024) - I didn’t go into this movie with any expectations, and I immensely enjoyed it.

    Next week

    I’m solo-parenting next weekend. I’m looking forward to hanging out with my kids and another weekend of being home.

    Saturday July 20, 2024

  • Django

    ,

    Office Hours

    ,

    Today I Learned

    📅 I am hosting Office Hours this Friday, July 19th

    I am hosting Office Hours this Friday, July 19th, 2024, from 2:00 pm to 4:15 pm Central Time.

    ➡️ I have to leave earlier than normal to pick up my kids, but I plan to be around until 4:15 pm to 4:30 pm Central Time or as time permits.

    ℹ️ Anyone can join office hours.

    ℹ️ Our office hours are a collaborative space where we can discuss our ongoing projects, catch up, and work together to wrap up our week on a productive note.

    💼 I will be working on a new Django project this week, sourcing data for it, and hopefully writing a bunch of views.

    🙏 As always, everyone is welcome to join, whether you’re a regular attendee or joining for the first time.

    ✅ If you need any additional details, feel free to send me a message or check out the gist from our previous sessions, where you’ll find the Zoom link ⚠️

    I look forward to seeing everyone.

    Wednesday July 17, 2024

  • Django

    ,

    Python

    🐘 Django Migration Operations aka how to rename Models

    Renaming a table in Django seems more complex than it is. Last week, a client asked me how much pain it might be to rename a Django model from Party to Customer. We already used the model’s verbose_name, so it has been referencing the new name for months.

    Renaming the model should be as easy as renaming the model while updating any foreign key and many-to-many field references in other models and then running Django’s make migrations sub-command to see where we are at.

    The main issue with this approach is that Django will attempt to create a new table first, update model references, and then drop the old table.

    Unfortunately, Django will either fail mid-way through this migration and roll the changes back or even worse, it may complete the migration only for you to discover that your new table is empty.

    Deleting data is not what we want to happen.

    As it turns out, Django supports a RenameModel migration option, but it did not prompt me to ask if we wanted to rename Party to Customer.

    I am also more example-driven, and the Django docs don’t have an example of how to use RenameModel. Thankfully, this migration operation is about as straightforward as one can imagine: class RenameModel(old_model_name, new_model_name)

    I re-used the existing migration file that Django created for me. I dropped the CreateModel and DeleteModel operations, added a RenameField operation, and kept the RenameField operations which resulted in the following migration:

    from django.db import migrations


class Migration(migrations.Migration):

    dependencies = [
        ('resources', '0002_alter_party_in_the_usa'),
    ]

    operations = [
        migrations.RenameModel('Party', 'Customer'),
        migrations.RenameField('Customer', 'party_number', 'customer_number'),
        migrations.RenameField('AnotherModel', 'party', 'customer'),
    ]

    The story’s moral is that you should always check and verify that your Django migrations will perform as you expect before running them in production. Thankfully, we did, even though glossing over them is easy.

    I also encourage you to dive deep into the areas of the Django docs where there aren’t examples. Many areas of the docs may need examples or even more expanded docs, and they are easy to gloss over or get intimidated by.

    You don’t have to be afraid to create and update your migrations by hand. After all, Django migrations are Python code designed to give you a jumpstart. You can and should modify the code to meet your needs. Migration Operations have a clean API once you dig below the surface and understand what options you have to work with.

    Monday July 15, 2024

  • Weeknotes

    ,

    Django

    ,

    Python

    ,

    Today I Learned

    📓 Weeknotes for Week 27: July 1 to 7

    The last week was a blur between the holiday, travel, and cramming a lot of work. My notes this week are more glossed over than most

    Family

    We drove to Illinois (near Chicago) to see family this week. We had family from both coasts who we don’t see very often, and it was the first time much of the family met my youngest, Nora. It takes us 8 to 8.5 hours to drive there, and people are always amazed at how good my kids are at traveling these distances. They both love the extra screen time and are easy to travel with. We can hand each one an iPad, and they are set for hours, given enough snacks and a rest stop every few hours.

    Work

    It was a short two-day week, but I got a bonus third day of work between fitting a few hours in the car and half a day on Friday. This was nice because we are wrapping up an existing client project while ramping up on another project.

    This has also made me realize that Django has many shopping cart projects, but they all seem to be outdated. We struggled to find a project we could use, which made me realize that the Django community is sorely missing a good shopping cart and checkout experience that works.

    Community Work

    Djangonaut Space: All three Djangonaut Space Team Neptune members have made meaningful contributions early in the process, and now I’m feeling the pressure to detail and share some more advanced projects so they can have tasks for the duration of the project.

    Django Code of Conduct WG: We have prioritized keeping up as we get new members used to contributing. This week, was a busy week.

    Side projects

    Django News Newsletter: We shipped issue #240.

    Django News Jobs: We have had code to aggregate some other Django job boards, and this week, we started allowing those to come through.

    Writing

    This week, I was back on track, and I wrote and published something every day. It was my first day of giving myself a buffer so that I am always writing tomorrow’s post, which took the edge and pressure off. I am still writing every day, but with the holiday, it was nice knowing I didn’t have to fit it in on our last day in Chicago, which ended up being a really long, full day with family.

    2024-07-07: 🧰 More fun with Django Extensions using shell_plus and graph_models Yesterday, I wrote about Django Extensions show_urls management command because it’s useful. I …

    2024-07-06: Django Extensions is useful even if you only use show_urls Yes, Django Extensions package is worth installing, especially for its show_urls command, which can …

    2024-07-05: 📅 No Office Hours on July 5th, but… No Office Hours this week (July 5th), but we will return next Friday, July 12th, 2024, 2:30 pm to …

    2024-07-04: 🎆 🤖 Happy AIndependence Day To everyone in the United States, Happy Independence Day and Happy AIndependence Day to everyone …

    2024-07-03: 🗳️ PSF Elections how I am voting This was written while driving to Chicago (technically from the passenger seat). Still, a few people …

    2024-07-02: 💬 On the PSF Bylaw changes The Python Software Foundation has three bylaw changes up for a vote in this year’s election. …

    2024-07-01: 📓 Weeknotes for Week 26: June 24 to 30 Family I took two days off for a funeral and some other family stuff. I saw cousins and other …

    Entertainment

    📺 Sweet Tooth - I really liked this series.

    📺 Presumed Innocent - I watched the first five episodes, and it’s okay.

    📺 The Last Thing He Told Me - I just started this series on Sunday night and am unsure if I’ll finish it.

    Next week

    It’s my first whole week back in a while. I’m looking forward to catching up and mowing my yard while listening to podcasts. I need to catch up on house stuff, including running my homemade above-ground sprinkler system. (Chill; we live near the Kansas River.)

    Monday July 8, 2024

  • Django

    🧰 More fun with Django Extensions using `shell_plus` and `graph_models`

    Yesterday, I wrote about Django Extensions show_urls management command because it’s useful. I have Mastodon posted/tooted about it [previously](https://mastodon.social/@webology/110271223054909764, but I didn’t expect it to possibly lead to it being added to Django, and yet here we are. My favorite byproduct of blogging is when someone talks about something they like, and someone asks, “What if” or “Why doesn’t?” and then they get inspired to look into it and contribute. This post might have led to one new contribution to Django. 🎉

    Several people shared that they also liked Django Extensions shell_plus and graph_models management commands.

    I don’t use shell_plus often, but I bake it into my Just workflows for clients who do. I tend to forget about it, and I spend so much time using pytest.set_trace() and testing.

    If you haven’t used graph_models, I use it in most of my client projects. I generate SVG files with it and add them to an ERD section of their docs, which helps discuss models and onboard new developers. It’s a nice-to-have feature and is a small lift with a huge payoff. This code is also easy to copy and paste from project to project.

    Sunday July 7, 2024

  • Django

    ,

    Python

    Django Extensions is useful even if you only use show_urls

    Yes, Django Extensions package is worth installing, especially for its show_urls command, which can be very useful for debugging and understanding your project’s URL configurations.

    Here’s a short example of how to use it because I sometimes want to include a link to the Django Admin in a menu for staff users, and I am trying to remember what name I need to reference to link to it.

    First, you will need to install it via:

    pip install django-extensions

# or if you prefer using uv like me:
uv pip install django-extensions

    Next, you’ll want to add django_extensions to your INSTALLED_APPS in your settings.py file:

    INSTALLED_APPS = [
    ...
    "django_extensions",
]

    Finally, to urn the show_urls management command you may do some by running your manage.py script and passing it the following option:

    $ python -m manage show_urls

    Which will give this output:

    $ python -m manage show_urls | grep admin
...
/admin/	django.contrib.admin.sites.index	admin:index
/admin/<app_label>/	django.contrib.admin.sites.app_index	admin:app_list
/admin/<url>	django.contrib.admin.sites.catch_all_view
# and a whole lot more...

    In this case, I was looking for admin:index which I can now add to my HTML document this menu link/snippet:

    ... 
<a href="{% url 'admin:index' %}">Django Admin</a>
...

    What I like about this approach is that I can now hide or rotate the url pattern I’m using to get to my admin website, and yet Django will always link to the correct one.

    Saturday July 6, 2024

  • Django

    🎆 🤖 Happy AIndependence Day

    To everyone in the United States, Happy Independence Day and Happy AIndependence Day to everyone else.

    In the spirit of staying ahead of the game and ensuring our online security, I came across this article by Cloudflare, Declare your AIndependence: block AI bots, scrapers and crawlers with a single click.

    I’m already using Cloudflare on several domains, and I enabled the AI Scrapers and Crawlers blocking features on Django Packages, Django News Jobs, and several websites I managed.

    If you wrote a scraper for Django Packages and are impacted by this change, contact me, and let’s talk. We have an API, a better solution than spidering the website.

    Thursday July 4, 2024

  • Weeknotes

    ,

    Django

    ,

    Python

    ,

    Today I Learned

    📓 Weeknotes for Week 26: June 24 to 30

    Family

    I took two days off for a funeral and some other family stuff. I saw cousins and other relatives who I only see at funerals, which is starting to feel like a habit.

    Work

    I was mentally blocked on a project, or at least the part I was putting off returning to. Something finally clicked over the weekend, and I worked through it. I hate working the weekends, which our company frowns upon. I didn’t expect to be out two days this week, and I’m out for the holiday most of the next week, so I tried to get caught up and ahead.

    Community Work

    Djangonaut Space: Week 2 kicked off, and I had to push our weekly standup back a week due to the funeral. No idea, but our Djangonauts are doing good work and working through their first issues.

    Side projects

    Django News Newsletter: We shipped issue #239.

    Django News Jobs: I had a weird issue that forced me to downgrade Python 3.12 to 3.11 to fix something internally that broke Pydantic. I’ll switch back in a few weeks, but I’m still annoyed by it.

    Overall, I just treaded water this week.

    Writing

    I shifted my writing schedule around this weekend so that I am a day ahead. I want to get four or five days ahead so that I have a nice publishing schedule to work with. I still plan on writing daily, but I like having more wiggle room.

    2024-06-30: 🐳 Using Just and Compose for interactive Django and Python debugging sessions When I wrote REST APIs, I spent weeks and months writing tests and debugging without looking at the …

    2024-06-29: 🐘 Docker Postgres Autoupgrades Upgrading Postgres in Docker environments can be daunting, but keeping your database up-to-date is …

    2024-06-28: 🐘 A Just recipe to backup and restore a Postgres database I have used this casey/just recipe to help backup and restore my Postgres databases from my Docker …

    2024-06-27: 📅 Office Hours for June 28th It’s been a week, but I’m hosting Office Hours this Friday at 2:30 pm. I was on the …

    2024-06-25: 🐳 Managing Docker Compose Profiles with Just: Switching Between Default and Celery Configurations For a recent client project, we wanted to toggle between various Docker Compose profiles to run the …

    2024-06-24: 🚜 Mastodon Bookmark exporter to Markdown/Frontmatter I wrote a Mastodon Bookmark exporter tool over the weekend and decided to polish it up and release …

    Entertainment

    📺 Presumed Innocent

    📺 Baby Reindeer

    📺 Dark Matter

    📺 The Acolyte - We got caught up, and OMG, it’s good.

    📺 The Flash (2023) - I couldn’t care less for the Flash, but I really enjoyed seeing Michael Keaton’s Batman again.

    Next week

    We are spending our holiday in Illinois to see family. It’ll be the first time my youngest has met several family members flying in for the holiday.

    Monday July 1, 2024

  • Django

    ,

    Python

    ,

    Justfiles

    ,

    Docker

    ,

    Today I Learned

    🐳 Using Just and Compose for interactive Django and Python debugging sessions

    When I wrote REST APIs, I spent weeks and months writing tests and debugging without looking at the front end. It’s all JSON, after all.

    For most of my projects, I will open two or three tabs. I’m running Docker Compose in tab one to see the logs as I work. I’ll use the following casey/just recipe to save some keystrokes and to standardize what running my project looks like:

    # tab 1
$ just up

    In my second tab, I’ll open a shell that is inside my main web or app container so that I can interact with the environment, run migrations, and run tests.

    We can nitpick the meaning of “console” here, but I tend to have another just recipe for “shell” which will open a Django shell using shell_plus or something more interactive:

    # tab 2
$ just console

    In my third tab, I’ll run a shell session for creating git branches, switching git branches, stashing git changes, and running my linter, which I prefer to run by hand.

    # tab 3
$ echo "I'm boring"

    Over the last year or two, the web has returned to doing more frontend work with Django and less with REST. Using ipdb, in my view, to figure out what’s going on has been really helpful. Trying to get ipdb to “just work” takes a few steps in my normal workflow.

    # tab 1 (probably)

# start everything
$ just start

# stop our web container
$ just stop web

# start our web container with "--service-ports" 
# just start-web-with-debug

    The only real magic here is using Docker’s --service-ports, which opens ports so we may connect to the open ipdb session when we open one in our view code.

    My main justfile for all of these recipes/workflows looks very similar to this:

    # justfile
set dotenv-load := false

@build *ARGS:
    docker compose build {{ ARGS }}

# opens a console
@console:
    docker compose run --rm --no-deps utility/bin/bash

@down:
    docker compose down

@start *ARGS:
    just up --detach {{ ARGS }}

@start-web-with-debug:
    docker compose run --service-ports --rm web python -m manage runserver 0.0.0.0:8000

@stop *ARGS:
    docker compose down {{ ARGS }}

@up *ARGS:
    docker compose up {{ ARGS }}

    If you work on multiple projects, I encourage you to find patterns you can scale across them. Using Just, Make, shell scripts or even Python lightens the cognitive load when switching between them.

    Sunday June 30, 2024

  • Weeknotes

    ,

    Django

    ,

    Python

    Weeknotes for Week 25: June 17 to 23

    Last week, my side of the family drove out to Colorado Springs, Colorado, and we rented a house for the week. The trip was great; seeing family was good, and watching the kids spend time with family they don’t get to see very often was the highlight for me. The mountains were great, but I don’t want to ride a train from 8k to 14k feet of altitude soon.

    Family

    The last of my biological aunts and uncles passed away two nights before we left Colorado. We had some other unexpected news, and I hope we have better news after the funeral this week.

    Work

    I took the whole week off, so the only work this week was community work.

    Community Work

    Djangonaut Space: Week 1 kicked off, and we had our first team meeting. The meeting went well, and I have a follow-up meeting on Monday with one Djangonaut who couldn’t make it.

    General Python: I had a ton of messages from Python friends this week asking for advice on upcoming bylaws changes. I wrote up a few drafts, and I might get around to publishing one or the other.

    Side projects

    Django News Newsletter: We shipped issue #238.

    Django Packages: More small updates and more on this sooner.

    I wrote a tool to export my Mastodon bookmarks. I tend to revisit these every day, and I wanted a way to create checklists for them and save them. I might find good content from the newsletter or posts I want to follow up on once I have more time.

    Writing

    I wrote every day but only published for three days, including today. I wrote several drafts, but I mostly spent time with my family.

    2024-06-19: 💜 Follow Black Python Devs on Juneteenth If you haven’t heard about Black Python Devs, Juneteenth is an excellent day to learn about …

    2024-06-17: Djangonaut Space has begun This is my first time helping out with Djangonaut Space as a Navigator for the Django Packages…

    Entertainment

    📺 Leave the World Behind - I rewatched half of this movie with my mother and sister.

    📺 Hellboy - I didn’t realize I had missed this one. I liked it even if I don’t know much about the Hellboy story outside the other movies.

    Next week

    I am returning to work a day earlier than planned, but it will be a long, short week.

    Sunday June 23, 2024

  • Django

    ,

    Djangonaut Space

    Djangonaut Space has begun

    This is my first time helping out with Djangonaut Space as a Navigator for the Django Packages project.

    Last week was week zero, making this week one and the first week my team gets to meet as a group.

    Coordinating meeting times is always hard when everyone is from a different timezone, and this was no exception. Thankfully, we found a 30-minute block that works for everyone.

    Everyone on my team contributed something positive immediately after the meeting invite, including a few project contributions. All in week zero!

    Overall, I’m looking forward to it. I feel under and over-prepared, but I keep getting reassured to take it slow. So, I have some new issues to tag and fill in details before we meet on Wednesday, but we have started, and that’s exciting.

    Monday June 17, 2024

  • Django

    🤖 More Blocking Bots with Django ❌

    Tonight, I ran across Robb Knight’s Blocking Bots with Nginx, which fits well with two pieces I have already written about.

    Check out my 🤖 Super Bot Fight 🥊 article, which develops a Django-friendly middleware for blocking AI UserAgents so they never get to your content.

    My 🤖 On Robots.txt article shared more of my research on the robots.txt standard.

    I forgot I wrote the middleware, so I rewrote it tonight before rediscovering it while looking for both articles to share. That was fun, so I’m not writing more tonight. 😬

    Friday June 14, 2024