Product sets allow you to create a new product for each option, with its own inventory management. With custom fields, on the other hand, you create a single product linked to the corresponding inventory. You can create several options for the product in question, but not separate products.
For example, let's say you have four sizes (S, M, L and XL) of a given product. When you use product sets, a separate inventory item is created for each size. If a customer buys a product in size S, the item is deducted from the size S inventory. Every size constitutes a separate product.
When you use custom fields, you have a single product linked to the a single inventory. It doesn't matter whether a customer buys a product in size S or XL; they are deducted from the same inventory.
Product sets are listed in a drop-down menu. If you want them presented in another manner—for example, as text fields, check boxes or radio buttons—use custom fields.