Portmaster is not just one process, but consists of multiple parts:
If you close the app, you only close your “window” into what Portmaster (the Core Service) is currently doing. In order to fully shut down Portmaster and have it stop blocking anything, you must select the option in either the app or the tray:
UI - Settings Page:
Tray Icon:
When you close the app for the first time, it clarifies what is and is not happening when closing the app.
Read more about the architecture of Portmaster over here.