“The Office” can be used to mean any job or typical day.  You do not actually have to work in an office.
Example:
John comes home tired, looking stressed and burned out.  His wife asks:  “Hard day at the office?”
He replies:  “Oh yeah.  We’ve got a new manager/supervisor/accountant/secretary/etc. causing me grief because…”
Once again, John doesn’t necessarily have to work at an office.  It can be any job, white collar or blue collar, volunteer, etc. that he is coming home from.

I have also heard that there is a bar/pub called “The Office” so that when your wife or husband asks “where are you now?” you can honestly say “I’m (still) at the office!”

“Office Politics” refers to power positioning at your place of employment and the perceived route to career success and promotion.  You have to have a good relationship with those in a position over your career.  It also refers to the idea that you should get along with others at work, and not ‘burn any bridges’ (damage any personal or professional relationships).