Definitions
Let γ be the total number of species in both assemblages.
Let α1 be the number of species in assemblage one.
Let α2 be the number of species in assemblage two.
Let Δα = α1 – α2.
Let <α> be the average of α1 and α2.
Let β = γ – <α>.
Let ν be the number of shared species (i.e., the number of species in both assemblages).
Assumption
All of the γ species are in assemblage 1, assemblage 2, or both.
Main result (Eq. 1)
For a given γ, Δα and β, the total number of unique assemblage pairs is [γ!] / [(γ – 2β)!(0.5(2β + Δα))!(0.5(2β – Δα))!].
Deriving this result is much easier once we have three simpler results.
Simpler result 1
X! is the total number of ways to order X objects.
Simpler result 2
(γ – 2β) is the number of shared species, ν.
Derivation of simpler result 2
Given that all species are in at least one of the two assemblages (see assumption above), it follows from the inclusion-exclusion principle that γ = α1 + α2 – ν. We also have that β = γ – 0.5(α1 + α2) from the definitions of β and <α>. Eliminating the alphas and solving for ν leads to simpler result 2.
Corollary of simpler result 2
2β = α1 + α2 – 2ν is the total number of unshared species (i.e., species that are only in either assemblage 1 or assemblage 2 but not in both).
Simpler result 3
0.5(2β + Δα) and 0.5(2β – Δα) are the numbers of unshared species in assemblages 1 and 2, respectively.
Derivation of simpler result 3
We derive only the first claim; the second follows immediately after the first. Note that we can write Δα = (α1 – ν) – (α2 – ν), from the definition of Δα, and 2β = (α1 – ν) + (α2 – ν), from the corollary of simpler result 2. Eliminating α2 – ν and solving for α1 – ν we obtain α1 – ν = 0.5 (Δα + 2β). Simpler result 3 follows once we recognize that α1 – ν is the number of unshared species in assemblage 1.
Derivation of main result
It follows from simpler result 1 that the numerator in Eq. 1 is the number of ways that γ species can be ordered. From Fig. 1, we see that the numerator gives the total number of ways that iterations from the simulations could occur. However, many of these ways result in identical assemblage pairs. This is because the order that species are listed in Fig. 1 determines a particular assemblage pair, only insofar as it determines whether each species is in assemblage 1 only, assemblage 2 only or in both assemblages 1 and 2. Therefore we must divide by the total number of ways to order the species within each of these three groups. It follows from simpler results 1 through 3 that the denominator in Eq. 1 gives this number.