The documentation of ${name##pattern} states that it prefers the longest match. I infer that the three following examples show produce the same output, namely "B":
% s=AB; echo ${s##(A)}
B
% s=AB; echo ${s##(A|)}
B
% s=AB; echo ${s##(|A)}
AB
It looks as if in the third case, after matching the empty string in "(|A)", the pattern matcher fails to backtrack to see if a longer match could be found.
I stumbled on this bug some weeks ago. I thought that it was with ${name#pattern}, which prefers the shortest match, but now I couldn't find an example that fails with it. Maybe I just misremember.
A similar test with ${name#pattern} works in all three cases:
% s=AA; echo ${s#()A}
A
% s=AA; echo ${s#(|A)A}
A
% s=AA; echo ${s#(A|)A}
A
Here, in the third case, after matching the "A" in "(A|)", the pattern matcher correctly backtracks to find the shortest match.
Philippe