There's a basic difference between an enum and a bitmask (option). You use an enum to list exclusive states. A bitmask is used when several properties can apply at the same time.

In both cases you use integers, but you look at them differently. With an enum you look at the numerical value, with bitmasks you look at the individual bits.

typedef NS_ENUM(NSInteger, MyStyle) {
MyStyleDefault,
MyStyleCustom
};
Will only represent two states. You can simply check it by testing for equality.

switch (style){
case MyStyleDefault:
// int is 0
break;
case MyStyleCustom:
// int is 1
break;
}
While the bitmask will represent more states. You check for the individual bits with logic or bitwise operators.

typedef NS_OPTIONS(NSInteger, MyOption) {
MyOption1 = 1 << 0, // bits: 0001
MyOption2 = 1 << 1, // bits: 0010
};

if (option & MyOption1){ // last bit is 1
// bits are 0001 or 0011
}
if (option & MyOption2){ // second to last bit is 1
// bits are 0010 or 0011
}
if ((option & MyOption1) && (option & MyOption2)){ // last two bits are 1
// bits are 0011
}
tl;dr An enum gives names to numbers. A bitmask gives names to bits.