Where/how are categories stored?

Disclaimer: I am new to WordPress.

When I add categories to my blogging site, where/how are these categories stored? If I want to change the names of these categories in batch, without having to go through the point-and-click UI, how would I go about it? Would this be done through a command-line interface?

There is extensive documentation on the database structure of WordPress in the Codex.
Its pretty simple but still you probably do not want to mess with the data directly.

As you mentioned CLI I’d recommend WP-CLI which offers some commands you need. A quick example from the docs to change the name of the term with the ID 15 from the taxonomy category to `Apple:

wp term update category 15 --name=Apple