Quick answer
Make a rule for each group and apply it to everyone in that group. Couples who are married, engaged or living together are invited together. Decide on children and coworkers as categories, not person by person, and you will rarely have to defend a single choice.
Rules beat case-by-case
Every awkward guest list conversation is easier when the answer is a rule rather than a judgment about one person. "We are inviting partners of everyone who is married, engaged or living together" is a sentence you can say to anyone. "We decided your boyfriend did not make the cut" is not.
Plus-ones
Invite established couples together, whether or not you have met the partner. Beyond that, offer a plus-one to guests who will not know anyone else at the wedding, and to members of the wedding party. Write the guest's name and their partner's on the invitation; "and guest" only where you are offering an open plus-one.
Children
Choose one of three clear options: all children welcome, only immediate family's children, or adults only. If you choose adults only, say so kindly on the invitation and website, and make an exception only for babies who are nursing, if at all. Expect some parents to decline and do not take it personally.
Coworkers
Invite your whole immediate team or none of it; inviting half a team causes more trouble than it is worth. Your manager is optional. Coworkers you see socially outside work count as friends, not coworkers.
People who invited you
You are not obliged to invite everyone whose wedding you attended, especially if years have passed or the guest counts differ. A friendship that is still close will make the list on its own merits.
Exes and complicated family
Invite an ex only if you are genuinely friends now and both of you are comfortable. For family members who do not get along, invite both, seat them apart and brief someone you trust to keep an eye on things.
Frequently asked questions
Do we have to give every guest a plus-one?
No. Invite established couples together, and offer a plus-one to guests who will not know anyone else and to the wedding party. Everyone else can be invited on their own.
How do we tell guests the wedding is adults only?
Address invitations only to the adults invited, and say it kindly on your wedding website, for example: we love your little ones, but our celebration is adults only.
Should we invite coworkers to our wedding?
Invite your whole immediate team or none of it. Coworkers you see socially outside work can be treated as friends.
Do we have to invite people whose weddings we went to?
No. Invite people because you are close now, not to return a favor from years ago.