While working on all kinds of Flash projects I've seen many different ways of masking the irises – some using a duplicate version of the eye whites symbol without the outlines or two copies of the same frames on two separate layers. If a symbol is used as a mask, Flash only reads the shapes that are placed on the bottom layer of that symbol. The other layers are ignored. I've never seen this documented as a feature. But it doesn't seem to be a bug either.

All methods that require the animator to deal with 2–3 separate instances – the eye whites and the mask (and sometimes the outline) make adding new frames, i.e. eye expressions a somehow complicated and non-intuitive process.

The Double Thickness method

I always prefer to work with lines instead of fills outlines whenever possible. It makes modifications and shape tweening a lot faster and easier to do. Of course, if thick-and-thin outlines are required this does not work.

In the example above the same symbol was used for the eye whites and the iris mask. The outline is twice as heavy as the other outlines, because only the portion that appears outside of the mask is actually visible in the composite image.
The fill of that same symbol is used as the iris mask and a white color card is used as underlay.

It may sound complicated, but is actually one of the simplest and most elegant masking tricks I've come up with. Download the sample file to study how it was made.

The Lines to Fills method

11 March 2020
Another method of masking is demonstrated in the video below:

Fills render better than Strokes, so this method provides some advantage in terms of output quality.

It's most serious disadvantage is that internally we need to maintain at least two layers: one for the mask and another one for the outline, because only the bottom layer in a Graphic symbol works as a mask. In most cases we would also want to keep the original artwork before converting lines to fills in order to be able to use it as a starting point for modifications, so this means a third guided layer inside.
And last, but not least, since fills become much more complex shapes, there will be complications with Shape Tweens.
On top of it all, Animate has additional issues (bugs) with such shape tweened masks, which are not present in Flash.

The Invisible Fill method

21 October 2023

Probably the best method so far.

It builds upon the Double Thickness method, but the outlines here are normally thick.
The trick is that the fill is set to 0% alpha. Then on the outside this is reversed with an Advance Color Effect to make it opaque.
The unmodified instance with the transparent fill is used on a top layer as an outline which covers the slightly larger mask.
Underneath an instance serves as eye whites. We apply an Advanced Color Effect of Alpha +255 here and this reveals the full color.

We can control the 3 instances with Smart Graphic Control panel with ease.

The benefits over the other two methods are that:

• we deal with only one layer inside, which makes edits very easy
• we keep the outlines as Stroke
• shape tweening works well
• line thickness is normal, so we see what we get after masking
• we can even have invisible gradients