Skip to content

8355719: Reduce memory consumption of BigInteger.pow() #24690

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 62 commits into from
Closed
Show file tree
Hide file tree
Changes from 58 commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
98a5b53
Add nthRoot(int) methods and optimize pow(int)
fabioromano1 Apr 16, 2025
926970e
Merge branch 'openjdk:master' into BigInteger-nth-root
fabioromano1 Apr 16, 2025
a3f1489
Remove trailing whitespaces
fabioromano1 Apr 16, 2025
d280c37
Correct loop recurrence according to proof of convergence
fabioromano1 Apr 16, 2025
91c3d1a
Correct initial estimate of nth root for BigIntegers
fabioromano1 Apr 17, 2025
1cfb775
Removed trailing whitespace
fabioromano1 Apr 17, 2025
c25bd32
Avoid an overflow in computing nth root estimate
fabioromano1 Apr 17, 2025
9c32ce4
Merge branch 'BigInteger-nth-root' of https://github.com/fabioromano1…
fabioromano1 Apr 17, 2025
7ff919b
optimize division in loop iteration of nth root
fabioromano1 Apr 17, 2025
1f5a9b4
An optimization
fabioromano1 Apr 17, 2025
b6c3320
Format code
fabioromano1 Apr 17, 2025
5d971fa
Correct left shift if shift is zero
fabioromano1 Apr 18, 2025
e459c23
Memory usage optimization
fabioromano1 Apr 18, 2025
48650de
An optimization
fabioromano1 Apr 18, 2025
54ec8f8
BigIntegers nth root's initial estimate optimization
fabioromano1 Apr 18, 2025
3ea5190
Extend use cases of MutableBigInteger.valueOf(double)
fabioromano1 Apr 19, 2025
6c9b364
An optimization
fabioromano1 Apr 19, 2025
3ca7d29
An optimization
fabioromano1 Apr 19, 2025
4516d88
Format code
fabioromano1 Apr 19, 2025
b427091
Format code
fabioromano1 Apr 19, 2025
524f195
Merge branch 'openjdk:master' into BigInteger-nth-root
fabioromano1 Apr 19, 2025
21fbf27
Code simplification
fabioromano1 Apr 20, 2025
e11b32f
Added reference for proof of convergence in the comment
fabioromano1 Apr 21, 2025
b527fa2
Revert format code changes
fabioromano1 Apr 21, 2025
8de6b82
Merge branch 'BigInteger-nth-root' of https://github.com/fabioromano1…
fabioromano1 Apr 21, 2025
00365c9
Merge branch 'BigInteger-nth-root' of https://github.com/fabioromano1…
fabioromano1 Apr 21, 2025
100d0e1
Merge remote-tracking branch 'origin/BigInteger-nth-root' into BigInt…
fabioromano1 Apr 22, 2025
1942fd1
Suggested change
fabioromano1 Apr 22, 2025
0e1a99e
Delete useless folder
fabioromano1 Apr 22, 2025
9cd136c
Optimized computation of nth root's remainder
fabioromano1 Apr 22, 2025
c3bd1b2
Format code
fabioromano1 Apr 22, 2025
f0d0605
An optimization
fabioromano1 Apr 22, 2025
f20d19b
Optimized BigInteger.pow(int) for single-word values
fabioromano1 Apr 24, 2025
8676af7
Optimized repeated squaring trick using cache for powers
fabioromano1 Apr 24, 2025
3cf820b
Some optimizations
fabioromano1 Apr 24, 2025
23914e8
Systematization of special cases in BigInteger.pow(int)
fabioromano1 Apr 24, 2025
bf099e4
Optimized nth root iteration loop
fabioromano1 Apr 26, 2025
f9bfd22
Merge branch 'openjdk:master' into BigInteger-nth-root
fabioromano1 Apr 26, 2025
7ceae87
Moved nth-root implementation to a dependent PR
fabioromano1 Apr 26, 2025
cb61ddc
Removed method used by nth-root
fabioromano1 Apr 26, 2025
10e122e
Put power's computation in a stand-alone method
fabioromano1 Apr 26, 2025
b94ca7e
Optimized BigInteger.pow(int) to support unsigned long bases
fabioromano1 Apr 26, 2025
139735d
Use BigInteger(long, int) constructor
fabioromano1 Apr 26, 2025
fcd5d55
Throw away unsignedIntPow(int, int)
fabioromano1 Apr 26, 2025
b831d01
Pre-cache the powers of x up to x^3 to simplify the code
fabioromano1 Apr 28, 2025
51272bc
Added tests for memory consumption
fabioromano1 Apr 28, 2025
70da95c
Decrease exponents in tests
fabioromano1 Apr 29, 2025
d85a634
Update test parameters
fabioromano1 Apr 29, 2025
9a5d696
Removed needless condition
fabioromano1 Apr 29, 2025
6033d25
Don't exclude a priori valid results
fabioromano1 Apr 29, 2025
5deb21a
Take into account special case exponent == 1
fabioromano1 Apr 29, 2025
280859b
Use a more loose formula to do range check
fabioromano1 Apr 29, 2025
925806b
Adjust the type of operand
fabioromano1 Apr 29, 2025
ad49b56
Use a more accurate formula to detect certain overflows
fabioromano1 Apr 29, 2025
6895926
Simplified the formula for detecting overflows
fabioromano1 Apr 29, 2025
b8ca4fe
Simplify long power computing
fabioromano1 Apr 30, 2025
4103e49
Suggested changes
fabioromano1 May 7, 2025
5ebc16b
Removed needless brackets
fabioromano1 May 7, 2025
e0816d5
Code simplification
fabioromano1 May 7, 2025
009937b
Suggested changes
fabioromano1 May 7, 2025
2e08f77
Suggested changes
fabioromano1 May 8, 2025
261dd31
Code simplification
fabioromano1 May 8, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
157 changes: 74 additions & 83 deletions src/java.base/share/classes/java/math/BigInteger.java
Original file line number Diff line number Diff line change
Expand Up @@ -1246,6 +1246,16 @@ else if (val < 0 && val >= -MAX_CONSTANT)
return new BigInteger(val);
}

/**
* Constructs a BigInteger with magnitude specified by the long,
* which may not be zero, and the signum specified by the int.
*/
private BigInteger(long mag, int signum) {
assert mag != 0 && signum != 0;
this.signum = signum;
this.mag = toMagArray(mag);
}

/**
* Constructs a BigInteger with the specified value, which may not be zero.
*/
Expand All @@ -1256,16 +1266,14 @@ private BigInteger(long val) {
} else {
signum = 1;
}
mag = toMagArray(val);
}

int highWord = (int)(val >>> 32);
if (highWord == 0) {
mag = new int[1];
mag[0] = (int)val;
} else {
mag = new int[2];
mag[0] = highWord;
mag[1] = (int)val;
}
private static int[] toMagArray(long mag) {
int highWord = (int) (mag >>> 32);
return highWord == 0
? new int[] { (int) mag }
: new int[] { highWord, (int) mag };
}

/**
Expand Down Expand Up @@ -2589,116 +2597,99 @@ public BigInteger pow(int exponent) {
if (exponent < 0) {
throw new ArithmeticException("Negative exponent");
}
if (signum == 0) {
return (exponent == 0 ? ONE : this);
}
if (exponent == 0 || this.equals(ONE))
return ONE;

BigInteger partToSquare = this.abs();
if (signum == 0 || exponent == 1)
return this;

BigInteger base = this.abs();
final boolean negative = signum < 0 && (exponent & 1) == 1;

// Factor out powers of two from the base, as the exponentiation of
// these can be done by left shifts only.
// The remaining part can then be exponentiated faster. The
// powers of two will be multiplied back at the end.
int powersOfTwo = partToSquare.getLowestSetBit();
int powersOfTwo = base.getLowestSetBit();
long bitsToShiftLong = (long)powersOfTwo * exponent;
if (bitsToShiftLong > Integer.MAX_VALUE) {
reportOverflow();
}
int bitsToShift = (int)bitsToShiftLong;

int remainingBits;

// Factor the powers of two out quickly by shifting right, if needed.
if (powersOfTwo > 0) {
partToSquare = partToSquare.shiftRight(powersOfTwo);
remainingBits = partToSquare.bitLength();
if (remainingBits == 1) { // Nothing left but +/- 1?
if (signum < 0 && (exponent&1) == 1) {
return NEGATIVE_ONE.shiftLeft(bitsToShift);
} else {
return ONE.shiftLeft(bitsToShift);
}
}
} else {
remainingBits = partToSquare.bitLength();
if (remainingBits == 1) { // Nothing left but +/- 1?
if (signum < 0 && (exponent&1) == 1) {
return NEGATIVE_ONE;
} else {
return ONE;
}
}
}
// Factor the powers of two out quickly by shifting right.
base = base.shiftRight(powersOfTwo);
int remainingBits = base.bitLength();
if (remainingBits == 1) // Nothing left but +/- 1?
return (negative ? NEGATIVE_ONE : ONE).shiftLeft(bitsToShift);

// This is a quick way to approximate the size of the result,
// similar to doing log2[n] * exponent. This will give an upper bound
// of how big the result can be, and which algorithm to use.
long scaleFactor = (long)remainingBits * exponent;

// Use slightly different algorithms for small and large operands.
// See if the result will safely fit into a long. (Largest 2^63-1)
if (partToSquare.mag.length == 1 && scaleFactor <= 62) {
// Small number algorithm. Everything fits into a long.
int newSign = (signum <0 && (exponent&1) == 1 ? -1 : 1);
long result = 1;
long baseToPow2 = partToSquare.mag[0] & LONG_MASK;

int workingExponent = exponent;

// Perform exponentiation using repeated squaring trick
while (workingExponent != 0) {
if ((workingExponent & 1) == 1) {
result = result * baseToPow2;
}

if ((workingExponent >>>= 1) != 0) {
baseToPow2 = baseToPow2 * baseToPow2;
}
}
// See if the result will safely fit into an unsigned long. (Largest 2^64-1)
if (scaleFactor <= Long.SIZE) {
// Small number algorithm. Everything fits into an unsigned long.
final int newSign = negative ? -1 : 1;
final long result = unsignedLongPow(base.mag[0] & LONG_MASK, exponent);

// Multiply back the powers of two (quickly, by shifting left)
if (powersOfTwo > 0) {
if (bitsToShift + scaleFactor <= 62) { // Fits in long?
return valueOf((result << bitsToShift) * newSign);
} else {
return valueOf(result*newSign).shiftLeft(bitsToShift);
}
} else {
return valueOf(result*newSign);
}
return bitsToShift + scaleFactor <= Long.SIZE // Fits in long?
? new BigInteger(result << bitsToShift, newSign)
: new BigInteger(result, newSign).shiftLeft(bitsToShift);
} else {
if ((long)bitLength() * exponent / Integer.SIZE > MAX_MAG_LENGTH) {
if ((bitLength() - 1L) * exponent >= (long) MAX_MAG_LENGTH << 5) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if ((bitLength() - 1L) * exponent >= (long) MAX_MAG_LENGTH << 5) {
if ((bitLength() - 1L) * exponent >= 32L * MAX_MAG_LENGTH) {

or

Suggested change
if ((bitLength() - 1L) * exponent >= (long) MAX_MAG_LENGTH << 5) {
if ((bitLength() - 1L) * exponent >= (long) Integer.SIZE * MAX_MAG_LENGTH) {

Both variant are easier to read, more honest, and exactly as efficient as with the shift. The right-hand sides are compile-time constants, so they have no impact on runtime performance.

More generally, the runtime compilers are perfectly capable to optimize multiplications by constant powers of 2 and replace them with shifts, even if the other operand is not a constant.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rgiulietti What about (bitLength() - 1L) * exponent >= Integer.MAX_VALUE?

Copy link
Contributor

@rgiulietti rgiulietti May 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah right, but you probably want

Suggested change
if ((bitLength() - 1L) * exponent >= (long) MAX_MAG_LENGTH << 5) {
if ((bitLength() - 1L) * exponent > Integer.MAX_VALUE) {

I mean > rather than >=

Copy link
Contributor Author

@fabioromano1 fabioromano1 May 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah right, but you probably want

No, the sufficient condition to get the overflow is (bitLength() - 1L) * exponent + 1L > Integer.MAX_VALUE, which is equivalent to that one I wrote above.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At some point you proposed

(bitLength() - 1L) * exponent >= (long) MAX_MAG_LENGTH << 5

Given the value of MAX_MAG_LENGTH, which is 2^26, this is equivalent to

(bitLength() - 1L) * exponent >= 1L << 31

that is, to

(bitLength() - 1L) * exponent > Integer.MAX_VALUE

What am I missing?

Copy link
Contributor Author

@fabioromano1 fabioromano1 May 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The condition A := (bitLength() - 1L) * exponent + 1L > Integer.MAX_VALUE is more accurate, as it compares the bit length of the result, in fact B := (bitLength() - 1L) * exponent >= (long) MAX_MAG_LENGTH << 5 implies A, but A does not imply B. The BigIntegers can have a mag length up to MAX_MAG_LENGTH, but MAX_MAG_LENGTH * Integer.SIZE > Integer.MAX_VALUE.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK.

But then your original expression

(((bitLength() - 1L) * exponent) >>> 5) + 1L > MAX_MAG_LENGTH

was a bit too restrictive as well, right?

Copy link
Contributor Author

@fabioromano1 fabioromano1 May 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK.

But then your original expression

(((bitLength() - 1L) * exponent) >>> 5) + 1L > MAX_MAG_LENGTH

was a bit too restrictive as well, right?

On the contrary, it was too loose, as it admitted a bit length equal to MAX_MAG_LENGTH * Integer.SIZE == 2^31 > Integer.MAX_VALUE.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean, if B strictly implies A, then B is more restrictive (stronger).
Since B is equivalent to your original formulation, to me it means that it was more restrictive.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean, if B strictly implies A, then B is more restrictive (stronger). Since B is equivalent to your original formulation, to me it means that it was more restrictive.

Exactly.

reportOverflow();
}

// Large number algorithm. This is basically identical to
// the algorithm above, but calls multiply() and square()
// the algorithm above, but calls multiply()
// which may use more efficient algorithms for large numbers.
BigInteger answer = ONE;

int workingExponent = exponent;
final int expZeros = Integer.numberOfLeadingZeros(exponent);
int workingExp = exponent << expZeros;
// Perform exponentiation using repeated squaring trick
while (workingExponent != 0) {
if ((workingExponent & 1) == 1) {
answer = answer.multiply(partToSquare);
}
for (int expLen = Integer.SIZE - expZeros; expLen > 0; expLen--) {
answer = answer.multiply(answer);
if (workingExp < 0) // leading bit is set
answer = answer.multiply(base);

if ((workingExponent >>>= 1) != 0) {
partToSquare = partToSquare.square();
}
workingExp <<= 1;
}

// Multiply back the (exponentiated) powers of two (quickly,
// by shifting left)
if (powersOfTwo > 0) {
answer = answer.shiftLeft(bitsToShift);
}
answer = answer.shiftLeft(bitsToShift);
return negative ? answer.negate() : answer;
}
}

if (signum < 0 && (exponent&1) == 1) {
return answer.negate();
} else {
return answer;
}
/**
* Computes {@code x^n} using repeated squaring trick.
* Assumes {@code x != 0 && x^n < 2^Long.SIZE}.
*/
static long unsignedLongPow(long x, int n) {
if (x == 1L || n == 0)
return 1L;

if (x == 2L)
return 1L << n;

/*
* The method assumption means that n <= 40 here.
* Thus, the loop body executes at most 5 times.
*/
long pow = 1L;
for (; n != 1; n >>>= 1) {
if ((n & 1) != 0)
pow *= x;

x *= x;
}
return pow * x;
}

/**
Expand Down
Loading