How can I set up a category “overview” page?

I know I can use the Template Hierarchy to set up custom theme pages for terms inside a category, like these:

mysite.com/category/tech/              // category-tech.php
mysite.com/category/blog/              // category-blog.php
mysite.com/category/featured/          // category-featured.php

category.php does not seem to work for the overview page, but it would work for a page like mysite.com/category/tech/ if a more specific template file (category-tech.php) was not available. I want to create a “category overview” page (mysite.com/category/) which would show all the terms of the category taxonomy.

Similarly, I would also like to be able to do this with a custom taxonomy, such as mysite.com/books/ in this case:

mysite.com/books/fiction/
mysite.com/books/non-fiction/
mysite.com/books/science/

I do want this done dynamically, i.e. I don’t want to create a page called ‘category’ with a permalink category and then add the items to that page. I think it would work but that seems like a hack.

There is no default rewrite to a template for a category or taxonomy landing page. You are correct, that this can be done by creating a custom template and applying it to a page.