Categories from Podlove 'podcast' Post Type

I have everything working and I don’t want to break anything. I searched the Wordpress forum, but I think it best to ask here.

I enabled the Category module on Day one. I have the standard Post Types and the ‘podcast’ Post Type created when I installed Podlove. When I enabled the Category module it basically copies over every category that is assigned to the ‘Post’ Post Type. When I add a New Category, it is instantly available on the ‘podcast’ Post Type.

I would live to restrict Categories on the PodLove ‘podcast’ post type. Can I and if so, how?

Thanks in advance