Tests not giving good error info

Hi all! I am porting some code from a very old Julia (0.4.6) to 1.5, using the latest version of Juno (where previously I was using a similarly old version of Juno).

I’m noticing some weird things going on with tests. I am now “using Test” rather than “using Base.Test”. So far, it seems that @test suppresses the usual error info and replaces it with a generic “there was an error during testing”. This is far less useful than the previous behavior, especially in Juno, where the old behavior would result in nicely highlighted lines showing me at what point the error arose within each function in the call stack, plus the informative error message.

For example:

using Test
just_error() = error("Just an error.")
just_error()
@test just_error()

The first call to just_error there displays my custom error message (“just an error”), as intended. The second call, with @test before it, instead displays the following:

Test.FallbackTestSetException("There was an error during testing")
record(::Test.FallbackTestSet, ::Union{Test.Error, Test.Fail}) at Test.jl:737
do_test(::Test.ExecutionResult, ::Any) at Test.jl:520
top-level scope at trees_test.jl:15

This is VERY annoying, because right now a lot of my tests are encountering errors due to differences between Julia 0.406 and 1.5. Lots of these would be trivial to fix if only I was getting the right error message. Instead I’m getting this uninformative generic message telling me that there was an error.

Is there something I should be doing differently, to get the original error messages?

In an attempt to get the actual error messages, I tried re-writing some of my test functions to do the actual calculations outside of @test, and only use @test to check if the values are right. Here’s a toy example of that:

OLD TEST (toy example):

@test 1 == 1.0
@test 2 == 2.0
@test 3 == 3.0

NEW TEST (toy example):

v1 = 1 == 1.0
v2 = 2 == 2.0
v3 = 3 == 3.0
@test v1
@test v2
@test v3

Unfortunately, it seems like the compiler is too smart for this, and re-writes the new version to the old version – because the result is exactly the same, hiding any errors. I haven’t actually been able to replicate this with a toy example, though. For example, this code snippet, using the same “just_error” function earlier:

function test_this()
  v = just_error()
  @test v 
end 

test_this()

results in the desired custom error message, rather than the generic “There was an error during testing”. I’m not sure what makes this different from my actual examples.

It’s not very meaningful without the rest of my code, of course, but here’s an actual snippet where I see the problem:

function random_rects_test(n)
  for i=1:n
    srand(i)
    dims = rand(1:5)
    a = BSPTree(randomrect(dims), randomtree(dims, rand(1:4)))
    v1 = epsilonequal(a,a)
    v2 = epsilonequal(a, a + 0.000001, 0.01)
    v3 = !epsilonequal(a, a + 0.1, 0.01)
    @test v1
    @test v2
    @test v3
  end
end

random_rects_test(50)

I’m getting Test.FallbackTestSetException(“There was an error during testing”) for the line where v2 is calculated, which doesn’t have a @test. I can only infer that the compiler is re-writing things to get rid of the variable v2, putting the @test directly on that line instead.

In any case, what I would much prefer would be a way to get informative error info without a workaround like that – I would prefer not to have to modify many hundreds of tests! But if there is a way to get a workaround like this to work reliably, that would also be helpful.

Oh, by the way, a further disturbing curiosity – when I remove the @test macros from random_rects_test entirely, it runs without any error. When I step through the execution line by line, I see an error again. :frowning: I have no idea what’s going on there.

You probably should be porting it to 0.7 instead. Version 0.7 is the same as the 1.0 but with better error messages for porting old codes. If the code works in 0.7 (and, consequently, 1.0) it will work for any 1.x versions (with very rare exceptions because some bugs fixes were not backported).

3 Likes

Thanks. Better error messages for porting old code sounds good, but do you think it will address the issue with @test?

Is there an easy way to switch between Julia versions in Juno?

I tried downloading 0.7 and changing the Julia path in Juno to use it. After re-starting Atom, Atom asked to do some package re-building for compatibility reasons, which appeared to work fine. But now the REPL does not work. I see this:


Press Enter to start Julia. 

Hold on tight while we are installing some packages for you.
This should only take a few seconds...

 Resolving package versions...
ERROR: LoadError: Unsatisfiable requirements detected for package JuliaInterpreter [aa1ae85d]:
 JuliaInterpreter [aa1ae85d] log:
 ├─possible versions are: [0.1.1, 0.2.0-0.2.1, 0.3.0-0.3.2, 0.4.0-0.4.1, 0.5.0-0.5.2, 0.6.0-0.6.1, 0.7.0-0.7.26, 0.8.0-0.8.1] or uninstalled
 ├─restricted by julia compatibility requirements to versions: uninstalled
 └─restricted by compatibility requirements with Atom [c52e3926] to versions: [0.3.0-0.3.2, 0.4.0-0.4.1, 0.5.0-0.5.2, 0.6.0-0.6.1, 0.7.0-0.7.26, 0.8.0-0.8.1] — no versions left
   └─Atom [c52e3926] log:
     ├─possible versions are: [0.8.0-0.8.8, 0.9.0-0.9.1, 0.10.0-0.10.2, 0.11.0-0.11.3, 0.12.0-0.12.24] or uninstalled
     └─restricted to versions * by an explicit requirement, leaving only versions [0.8.0-0.8.8, 0.9.0-0.9.1, 0.10.0-0.10.2, 0.11.0-0.11.3, 0.12.0-0.12.24]
Stacktrace:
 [1] #propagate_constraints!#61(::Bool, ::Function, ::Pkg.GraphType.Graph, ::Set{Int64}) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/Pkg/src/GraphType.jl:1005
 [2] propagate_constraints! at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/Pkg/src/GraphType.jl:946 [inlined]
 [3] #simplify_graph!#121(::Bool, ::Function, ::Pkg.GraphType.Graph, ::Set{Int64}) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/Pkg/src/GraphType.jl:1460
 [4] simplify_graph! at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/Pkg/src/GraphType.jl:1460 [inlined] (repeats 2 times)
 [5] macro expansion at ./logging.jl:311 [inlined]
 [6] resolve_versions!(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/Pkg/src/Operations.jl:339
 [7] #add_or_develop#58(::Array{Base.UUID,1}, ::Function, ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/Pkg/src/Operations.jl:1163
 [8] #add_or_develop at ./none:0 [inlined]
 [9] #add_or_develop#13(::Symbol, ::Bool, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/Pkg/src/API.jl:64
 [10] #add_or_develop at ./none:0 [inlined]
 [11] #add_or_develop#12 at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/Pkg/src/API.jl:29 [inlined]
 [12] #add_or_develop at ./none:0 [inlined]
 [13] #add_or_develop#11(::Base.Iterators.Pairs{Symbol,Symbol,Tuple{Symbol},NamedTuple{(:mode,),Tuple{Symbol}}}, ::Function, ::Array{String,1}) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/Pkg/src/API.jl:28
 [14] #add_or_develop at ./none:0 [inlined]
 [15] #add_or_develop#10 at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/Pkg/src/API.jl:27 [inlined]
 [16] #add_or_develop at ./none:0 [inlined]
 [17] #add#18 at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/Pkg/src/API.jl:69 [inlined]
 [18] add(::String) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v0.7/Pkg/src/API.jl:69
 [19] top-level scope at /Users/abram/.atom/packages/julia-client/script/boot_repl.jl:21
 [20] include at ./boot.jl:317 [inlined]
 [21] include_relative(::Module, ::String) at ./loading.jl:1038
 [22] include(::Module, ::String) at ./sysimg.jl:29
 [23] exec_options(::Base.JLOptions) at ./client.jl:239
 [24] _start() at ./client.jl:432
in expression starting at /Users/abram/.atom/packages/julia-client/script/boot_repl.jl:1

Julia has exited.
Press Enter to start a new session.

Pressing enter again gets the same result. The same thing happens when trying to run code any other way.

Perhaps Juno is not compatible with 0.7?

looks like a typo since 0.7 should be equivalent to 1.0

I checked whether the path I put into Juno was correct. Here is the path for reference. I’m on a mac, so “installing” julia 0.7 means copying it into applications.

/Applications/Julia-0.7.app/Contents/Resources/julia/bin/julia

/Contents/Resources/julia/bin/julia is the path I got from this explanation of how to install Julia.

When I type /Applications/Julia-0.7.app/Contents/Resources/julia/bin/julia into the command line, it opens Julia 0.7 as expected. So I think the path is correct.

Is it possible there’s something else wrong, aside from a typo or an incompatibility?

Any other ideas on how to get the desired error info from tests would be appreciated.

I meant a typo in their [compat], not your local config or anything.

1 Like

Oh I see.

There have been several replies here, but no one has addressed my basic question. Is the new error-hiding behavior of @test the intended behavior? Is this normal, or something weird I’m seeing? Is there a way to change it? Is there some other workaround?

Thanks,
Abram

How do you run these tests? I see:

julia> @test just_error()
Error During Test at REPL[4]:1
  Test threw exception
  Expression: just_error()
  Just an error. # your error message <------------------
  Stacktrace:
   [1] error(::String) at ./error.jl:33
   [2] just_error() at ./REPL[2]:1
   [3] top-level scope at REPL[4]:1
   [4] eval(::Module, ::Any) at ./boot.jl:331
    ....

@kristoffer.carlsson I’m running the tests by running the lines in a file, rather than copying the tests into the REPL. And I was looking at the error output Juno displays next to the line, rather than the output in the REPL.

I now notice that in the just_error example I gave the “real” error output ends up displaying in the REPL. So that’s good. But in my actual examples in code that’s not happening! I’m just getting ERROR: There was an error during testing, much like I’m getting in Juno’s in-line display.

I’ll see what I can do to replicate the issue in a simple example. In the meantime, any thoughts are appreciated.

Thanks!

Clarification:

In the @test just_error() example,

(1) Running the test in-place by hitting command-enter, I see a red error report displayed after the line by Juno, which gives the uninformative “There was an error during testing” error. However, the real error is printed in the REPL. So that’s OK.

(2) Running the test in the REPL similarly results in the detailed error appearing in the REPL.

In my real problem case,

(1) Running the test by executing the line in-file in Juno, I just get the uninformative “There was an error during testing” rather than information about what the error is. The real information does not display in the REPL, unlike in the just_error example.

(2) Similarly, running my real example in the REPL directly just displays the “There was an error” error. Specifically, here’s my REPL output:

Test Failed at /Users/abram/Dropbox/Research/Julia/Signia/scratch/minimal_weird_case.jl:549
  Expression: v1
ERROR: There was an error during testing

No stack trace or anything.

I haven’t gotten a very minimal example together, but here is a code dump which demonstrates the problem. The relevant test is right at the end; everything else is just the code needed to make the example work.

Keep in mind I was in the process of porting this code to 1.5, so some stuff is wonky.


using Test
using Random

srand(x) = Random.seed!(x)

using JuMP
using Clp

import Base.==

abstract type Split end

struct LinSplit <: Split
  coefs::Vector{Float64}
  con::Float64
end

function ==(l1::LinSplit,l2::LinSplit)
  if l1.con!=l2.con
    return false
  else
    for i in 1:max(length(l1.coefs), length(l2.coefs))
      if get(l1.coefs, i, 0.0) != get(l2.coefs, i, 0.0)
        return false
      end
    end
    return true
  end
end

LinSplit(coef,con) =
  LinSplit(convert(Vector{Float64}, coef), convert(Float64, con))

abstract type BSPNode end

struct None <: BSPNode
end

struct SplitNode <: BSPNode
  split::Split # split definition
  neg::BSPNode # negative side
  pos::BSPNode # positive side
end

==(s1::SplitNode, s2::SplitNode) =
  s1.split == s2.split && s1.neg == s2.neg && s1.pos == s2.pos

struct LeafNode <: BSPNode
  value::Any
end

==(l1::LeafNode,l2::LeafNode) =
  l1.value == l2.value

abstract type Region end

struct LinBound
  split::LinSplit
  positive::Bool
end

==(l1::LinBound,l2::LinBound) =
  l1.split == l2.split && l1.positive == l2.positive

import Base.isless

function isless(l1::LinBound, l2::LinBound) # defining so they can be sorted
  if l1.positive < l2.positive
    return true
  elseif l2.positive < l1.positive
    return false
  elseif l1.split.con < l2.split.con
    return true
  elseif l2.split.con < l1.split.con
    return false
  else
    for i in 1:length(l1.split.coefs)
      if l1.split.coefs[i] < l2.split.coefs[i]
        return true
      elseif l2.split.coefs[i] < l1.split.coefs[i]
        return false
      end
    end
    return false
  end
end

struct LinRegion <: Region
  bounds::Vector{LinBound}
  dimensionality::Int
end

function regularize_bound(b::LinBound)
  z = sqrt(sum(b.split.coefs .^ 2))
  if z == 0.0
    return b
  else
    return LinBound(LinSplit(b.split.coefs / z, b.split.con / z), b.positive)
  end
end

function ==(l1::LinRegion,l2::LinRegion)
  b1 = map(regularize_bound, trim_region(l1).bounds)
  b2 = map(regularize_bound, trim_region(l2).bounds)
  return sort(b1) == sort(b2)
end

boundless_reg(d::Int) = LinRegion(Vector{LinBound}(),d)

function rect_split(dim,loc,numdims)
  z = zeros(numdims)
  z[dim]=1.0
  return LinSplit(z,loc)
end

function rect_region(mins, maxs)
  max_d = length(mins)
  s = Vector{LinBound}()
  for d in 1:max_d
    z = zeros(max_d)
    z[d] = 1.0
    s = union(s,[LinBound(LinSplit(z,mins[d]), true)])
  end
  for d in 1:max_d
    z = zeros(max_d)
    z[d] = 1.0
    s = union(s,[LinBound(LinSplit(z,maxs[d]), false)])
  end
  return LinRegion(s,max_d)
end

import Base.isequal

isequal(t1::SplitNode, t2::SplitNode) =
  isequal(t1.split, t2.split) &&
  isequal(t1.pos, t2.pos) &&
  isequal(t1.neg, t2.neg)

isequal(t1::LinSplit, t2::LinSplit) =
  t1.coefs == t2.coefs &&
  t1.con == t2.con

struct BSPTree
  boundary::Region
  root::BSPNode
end

==(t1::BSPTree, t2::BSPTree) = (t1.boundary == t2.boundary) && isequal(t1.root, t2.root)
isequal(t1::BSPTree, t2::BSPTree) = t1 == t2

function randomrect(n)
  mins = rand(n)
  maxs = rand(n)+mins
  return rect_region(mins,maxs)
end

function randomrect()
  randomrect(2)
end

function randomsplit(n)
  LinSplit(rand(n),rand())
end

function random_rect_split(n)
  coefs = zeros(n)
  coefs[rand(1:n)] = 1.0
  LinSplit(coefs,rand())
end

# creates a balanced tree of the given depth
function randomtree(dims, depth)
  if depth == 0
    return LeafNode(rand())
  else
    return SplitNode(randomsplit(dims),
                     randomtree(dims,depth-1),
                     randomtree(dims,depth-1))
  end
end

function random_rect_tree(dims, depth)
  if depth == 0
    return LeafNode(rand())
  else
    return SplitNode(random_rect_split(dims),
                     random_rect_tree(dims,depth-1),
                     random_rect_tree(dims,depth-1))
  end
end

function epsilonequal(t1::BSPTree, t2::BSPTree, epsilon::Float64=0.0000001)
  if !(regvalid(intersection(t1.boundary, t2.boundary), epsilon))
    return true # If they have no intersection, they trivially agree about it!
  elseif typeof(t1.root) == SplitNode
    return epsilonequal(takepos(t1), t2, epsilon) && epsilonequal(takeneg(t1), t2, epsilon)
  elseif typeof(t2.root) == SplitNode
    return epsilonequal(takepos(t2), t1, epsilon) && epsilonequal(takeneg(t2), t1, epsilon)
  elseif t1.root == None() && t2.root == None()
    return true
  elseif t1.root == None() || t2.root == None()
    return false
  else
    return epsilonequal(t1.root.value, t2.root.value, epsilon)
  end
end

epsilonequal(a::Float64, b::Float64, epsilon::Float64=0.0000001) = (a == b) || (abs(a - b) < epsilon)

epsilonequal(a::Number, b::Number, epsilon::Float64=0.0000001) = epsilonequal(convert(Float64,a), convert(Float64,b), epsilon)

epsilonequal(a::LinBound, b::LinBound, epsilon::Float64=0.0000001) =
  a.positive == b.positive && epsilonequal(a.split, b.split, epsilon)

epsilonequal(a::LinSplit, b::LinSplit, epsilon::Float64=0.0000001) =
  epsilonequal(a.con, b.con, epsilon) && all(map(((x,y) -> epsilonequal(x,y, epsilon)), a.coefs, b.coefs)) # shouldn't really be assuming the vectors have same length here

function intersection(r1::LinRegion, r2::LinRegion)
  return LinRegion(union(r1.bounds, r2.bounds), max(r1.dimensionality, r2.dimensionality))
end

function rect_valid(r::Region)
  max_d = r.dimensionality
  maxs = fill(Inf, max_d)
  maxs_closed = trues(max_d) # whether the maximum is closed
  mins = fill(-Inf, max_d)
  mins_closed = trues(max_d) # whether the minimum is closed
  for bound in r.bounds # Go through bounds checking number of nonzero coefs; if >1, return 0; if 1, record the new min/max implied.
    bound_coefs = bound.split.coefs
    bound_length = length(bound_coefs)
    if bound_length > max_d
      throw("rect_valid handed a region with more coefficients than dimensions: $r")
    end
    nonzero_count = 0
    nonzero_loc = 0
    for dim in 1:bound_length
      if !(bound_coefs[dim] == 0.0)
        nonzero_count = nonzero_count + 1
        if nonzero_count > 1
          return 0
        end
        nonzero_loc = dim
      end
    end
    if nonzero_loc > 0 #if the bound is doing anything at all
      # Now that we know the bound is uni-dimensional, add it to the list of dimension bounds.
      adjusted_con = bound.split.con/bound_coefs[nonzero_loc] # The actual location for this bound.
      if bound.positive # closed bound
        if bound_coefs[nonzero_loc] > 0.0 # positive coef, so bound.positive=true means the bound is expressing a minimum
          if adjusted_con > mins[nonzero_loc] # lower bound would be increased
            mins[nonzero_loc] = adjusted_con
            mins_closed[nonzero_loc] = true # inherits the closed status, since we increased past whatever the bound previously might have been
          end # if the bound is less than or equal to one already established, we have no need to update the open or closed status in this case, since it could only be loosening the constraint, which we don't want to do
        elseif bound_coefs[nonzero_loc] < 0.0 # negative coef, so bound is actually a maximum, since -x>=c means x<=c
          if adjusted_con < maxs[nonzero_loc] # existing upper bound would be decreased
            maxs[nonzero_loc] = adjusted_con
            maxs_closed[nonzero_loc] = true # inherits the open status, since we decreased past whatever the previous bound would have been
          end # no need to update open/closed status otherwise
        end
      else # open bound
        if bound_coefs[nonzero_loc] > 0.0 # positive coef, so bound is saying x<c, IE giving a max
          if adjusted_con < maxs[nonzero_loc] # new max would be lower than old max
            maxs[nonzero_loc] = adjusted_con # lower the max
            maxs_closed[nonzero_loc] = false # inherit the open status
          elseif adjusted_con == maxs[nonzero_loc] # if they're equal, we don't need to update the max, but we may need to switch it from closed to open since an open constraint is stricter
            maxs_closed[nonzero_loc] = false # let's just assign; no point checking
          end # otherwise, nothing to update; we were already maximally strict
        elseif bound_coefs[nonzero_loc] < 0.0 && adjusted_con >= mins[nonzero_loc] # negative coef, so we're saying -x<c, IE x>c, so we are giving a min
          if adjusted_con > mins[nonzero_loc] # new min would be higher
            mins[nonzero_loc] = adjusted_con
            mins_closed[nonzero_loc] = false # inherit the open status
          elseif adjusted_con == mins[nonzero_loc] # if they're equal, we still need to enforce openness
            mins_closed[nonzero_loc] = false
          end
        end
      end
    else # no nonzero locs in this bound, because we already checked whether there's 1, and returned for the case where there's more than one
      if bound.positive && bound.split.con > 0 # no way to get > 0 with no coefficients; this region is invalid
        return -1
      elseif !bound.positive && bound.split.con <= 0 # if the constant is 0 or below, there's no way we can get strictly under it
        return -1
      end
    end
  end
  # Now that we've added all the bounds, check for validity.
  for dim in 1:max_d
    if maxs[dim] < mins[dim] || (maxs[dim]==mins[dim] && !(maxs_closed[dim] && mins_closed[dim]))
      return -1
    end
  end
  return +1
end

function zero_bound_check(b::LinBound) # using the same idea as the zero-coef bound case in rect_valid, returns +1 if the bound rules out nothing, -1 if everything, otherwise zero
  if all((x->x==0.0), b.split.coefs)
    if b.positive
      if b.split.con > 0
        return -1
      else
        return +1
      end
    else
      if b.split.con <= 0
        return -1
      else
        return +1
      end
    end
  else
    return 0
  end
end


function regvalid(r::LinRegion, epsilon::Float64=0.0000001)
  r_v = rect_valid(r)
  if r_v==1
    return true
  elseif r_v==-1
    return false
  end
  m = makemodel(r)
  for bound in r.bounds
    if !(bound.positive)
      d = -bound.split.coefs
      @objective(m[1], Max, sum(d[i]*(m[2])[i] for i=1:length(d))) # optimize away from each face
      optimize!(m[1])
      s = termination_status(m[1])
      if s == :Infeasible # not even the region's closure is valid
        return false
      elseif s == :Optimal
        optimize_away = getobjectivevalue(m[1])
        difference = optimize_away + bound.split.con # Since things are negative, I want the optimized value minus the **negative of** the constant; so really I want to add them.
        if difference < epsilon
          return false
        end
      end
    end
  end
  return true
end

regvalid(r::BSPTree) =
  regvalid(r.boundary)


takepos(reg::LinRegion, split::LinSplit) = LinRegion(union(reg.bounds, [LinBound(split, true)]), reg.dimensionality)
takeneg(reg::LinRegion, split::LinSplit) = LinRegion(union(reg.bounds, [LinBound(split, false)]), reg.dimensionality)

takepos(tree::BSPTree) =
  BSPTree(takepos(tree.boundary, tree.root.split), tree.root.pos)

takeneg(tree::BSPTree) =
  BSPTree(takeneg(tree.boundary, tree.root.split), tree.root.neg)


function makemodel(r::LinRegion)
  m = Model(optimizer_with_attributes(Clp.Optimizer, "InfeasibleReturn" => 1, "LogLevel" => 0))
  d = r.dimensionality
  @variable(m,x[1:d])
  for bound in r.bounds
    if bound.positive
      @constraint(m, sum(x[i]*bound.split.coefs[i] for i=1:length(bound.split.coefs)) >= bound.split.con)
    else
      @constraint(m, sum(x[i]*bound.split.coefs[i] for i=1:length(bound.split.coefs)) <= bound.split.con)
    end
  end
  return (m, x)
end

function cheap_treearithmetic(operator, base_type)
  eval(
    quote

      function $operator(n1::BSPNode, n2::$base_type) # Only work on raw nodes if we're combining with a number; otherwise we want to track regions to avoid computing things which would be trimmed, since that can make an exponential difference.
        if typeof(n1) == LeafNode
          return LeafNode( $operator(n1.value, n2))
        elseif n1 == None()
          # return None()
          return LeafNode(n2) # None() combined with anything else acts like an identity element.
        elseif typeof(n1) == SplitNode
          return SplitNode(n1.split, $operator(n1.neg, n2),
                                     $operator(n1.pos, n2))
        end
      end

      function $operator(n1::$base_type, n2::BSPNode)
        if typeof(n2) == LeafNode
          return LeafNode( $operator(n1, n2.value))
        elseif n2 == None()
          #return None()
          return LeafNode(n1) # None() combined with anything else acts like an identity element.
        elseif typeof(n2) == SplitNode
          return SplitNode(n2.split, $operator(n1, n2.neg),
                                     $operator(n1, n2.pos))
        end
      end

      function $operator(t1::BSPTree, t2::BSPTree)
        i = intersection(t1.boundary, t2.boundary)
        if typeof(t1.root) == LeafNode
          if !regvalid(i)
            return BSPTree(i, None())
          end
          return merge_bottom_up(BSPTree(i, ($operator(t1.root.value, cheap_trim(t2, i).root))))
        elseif typeof(t1.root) == SplitNode
          pos = takepos(t1)
          neg = takeneg(t1)
          return BSPTree(i,
                         SplitNode(t1.root.split,
                                   identity($operator(neg, t2)).root, # I have to use "identity" here to prevent the order of operations from being changed when $operator is expanded.
                                   identity($operator(pos, t2)).root)) #
        elseif  t1.root == None()
          return t2
        elseif t2.root == None()
          return t1
        end
      end

      $operator(t1::BSPTree, t2::$base_type) = BSPTree(t1.boundary, $operator(t1.root, t2))
      $operator(t1::$base_type, t2::BSPTree) = BSPTree(t2.boundary, $operator(t1, t2.root))
    end
    )
end

cheap_treearithmetic(op) = cheap_treearithmetic(op, Number)


import Base.+
cheap_treearithmetic(:+)
import Base.*
cheap_treearithmetic(:*)
import Base.max
cheap_treearithmetic(:max)
import Base.min
cheap_treearithmetic(:min)
import Base./
cheap_treearithmetic(:/)
import Base.-
cheap_treearithmetic(:-)
import Base.^
cheap_treearithmetic(:^,Integer) # adding the integer definitions to eliminate some compiler warnings about ambiguous matches
cheap_treearithmetic(:^)


function random_rects_test(n)
  for i=1:n
    srand(i)
    dims = rand(1:5)
    a = BSPTree(randomrect(dims), randomtree(dims, rand(1:4)))
    v1 = epsilonequal(a,a)
    v2 = epsilonequal(a, a + 0.000001, 0.01)
    v3 = !epsilonequal(a, a + 0.1, 0.01)
    @test v1
    @test v2
    @test v3
  end
end

random_rects_test(50)