After some time, I managed to find a perhaps not-optimal but simple algorithm that proves that the complexity of the problem is polynomial.
Algorithm
(a) Compute a generating-set of the orthogonal subgroup H⊥ of H.
(b) Check whether or not the element b is orthogonal to H⊥.
There are efficient clasical algorithms for problems (a) and (b) (see analysis below). This gives an efficient membership-test since an element b is orthogonal to H⊥ if and only if h∈H.
Analysis
The orthogonal subgroup H⊥ is defined via the character group of G as:
H⊥:={g∈G:χg(h)=1∀h∈H}
Main properties:
- H⊥ is a subgroup of G.
- H⊥⊥=H
Algorithm for (a):
I follow an algorithm from [1] with minor variations. g belongs to H⊥ if and only if χg(h)=1 for all h∈H, but, by linearity it is enough to show χb(hi)=1 for each generator of H. Expanding the character in terms of exponentials (here I implicitly use the cyclic factor decomposition) this condition is equivalent to
exp{2πi(g(1)hi(1)d1+…+g(m)hi(m)dm)}=1
To solve these equations, compute
M:=lcm(N1,…,Nd) using the Euclidean algorithm and the numbers
αi:=M/di. We can re-write the conditions above for every
i as a system of linear modular equations.
⎛⎝⎜⎜⎜⎜α1h1(1)α1h2(1)⋮α1hn(1)α2h1(2)α2h2(2)⋮α2hn(2)……⋯…αmh1(m)αmh2(m)⋮αmhn(m)⎞⎠⎟⎟⎟⎟⎛⎝⎜⎜⎜⎜g(1)g(2)⋮g(n)⎞⎠⎟⎟⎟⎟=⎛⎝⎜⎜⎜⎜00⋮0⎞⎠⎟⎟⎟⎟modMmodM⋮modM
As it is proven in
1, if we sample
t+⌈log|G|⌉ random solutions of this system of equations we will obtain a generating set of
H⊥ with probability exponentially close to one
p≥1−1/2t. Now to sample from this equations write them in matrix form
AX=0(modM). Here
A is a rectangular matrix over the integers modulo
M for which an algorithm given in
2 allows to efficiently compute its Smith normal decomposition. The algorithm returns a diagonal matrix
D and two invertible matrices
U,
V such that
D=UAV. Using this formula the system of equations can be written as
DY=0(modM) with
X=VY. Now it is possible to randomly compute solutions of
DY=0(modM) using Euclid's algorithm, since this is a system of equations of the form
diyi=0(modM). Finally, computing
X=VY one obtains a random element of the orthogonal group
H⊥ as desired.
Algorithm for (b):
Since we already know how to compute a generating-set of H⊥, it is easy to check if a given element b belongs to H. First compute a generating-set ⟨g1,…,gs⟩ of H⊥. Then, by definition, b belongs to H if and only if χb(gi)=1 for all generators of H⊥. Since there are a O(polylog(|G|)) number of them and this can be done efficiently using modular arithmetic we are done.