You need to call something that ends up calling the pirated method, for example:
julia> @eval Base mapreduce_first(::typeof(identity), ::typeof(max), ::Int) = throw(ErrorException("piracy!"))
mapreduce_first (generic function with 2 methods)
julia> maximum([1])
ERROR: piracy!
Stacktrace: