Skip to content

[AArch64] Prepare for split ZPR and PPR area allocation (NFCI) #142391

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: users/MacDue/pred_stack_id
Choose a base branch
from
+269 −150
Open

[AArch64] Prepare for split ZPR and PPR area allocation (NFCI) #142391

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
8664a89
[AArch64] Prepare for split ZPR and PPR area allocation (NFCI)
MacDue May 8, 2025
eba3bca
Rename StackID
MacDue Jun 26, 2025
4e231e7
Fixups
MacDue Jul 8, 2025
1f4858f
Rebase
MacDue Jul 9, 2025
3d23b57
Fixups
MacDue Jul 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
314 changes: 203 additions & 111 deletions llvm/lib/Target/AArch64/AArch64FrameLowering.cpp
View file
Open in desktop

Large diffs are not rendered by default.

12 changes: 8 additions & 4 deletions llvm/lib/Target/AArch64/AArch64FrameLowering.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,11 @@

namespace llvm {

struct SVEStackSizes {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be named SVEStackOffsets (given that they're used as signed offsets)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The return value of determineSVEStackObjectOffsets() is the SVE stack sizes (the values will be >= 0).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, could you then change int64_t to an unsigned type?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's best to keep the value signed as the sizes are used to derive signed offsets, and mixing signed/unsigned can sometimes lead to unexpected results.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

e.g.: https://godbolt.org/z/hcWWco8Yj

int64_t ZPRStackSize{0};
int64_t PPRStackSize{0};
};

class AArch64FrameLowering : public TargetFrameLowering {
public:
explicit AArch64FrameLowering()
Expand Down Expand Up @@ -147,10 +152,8 @@ class AArch64FrameLowering : public TargetFrameLowering {
bool shouldCombineCSRLocalStackBump(MachineFunction &MF,
uint64_t StackBumpBytes) const;

int64_t estimateSVEStackObjectOffsets(MachineFrameInfo &MF) const;
int64_t assignSVEStackObjectOffsets(MachineFrameInfo &MF,
int &MinCSFrameIndex,
int &MaxCSFrameIndex) const;
SVEStackSizes estimateSVEStackObjectOffsets(MachineFunction &MF) const;
SVEStackSizes assignSVEStackObjectOffsets(MachineFunction &MF) const;
bool shouldCombineCSRLocalStackBumpInEpilogue(MachineBasicBlock &MBB,
uint64_t StackBumpBytes) const;
void emitCalleeSavedGPRLocations(MachineBasicBlock &MBB,
Expand All @@ -166,6 +169,7 @@ class AArch64FrameLowering : public TargetFrameLowering {
int64_t RealignmentPadding, StackOffset AllocSize,
bool NeedsWinCFI, bool *HasWinCFI, bool EmitCFI,
StackOffset InitialOffset, bool FollowupAllocs) const;

/// Make a determination whether a Hazard slot is used and create it if
/// needed.
void determineStackHazardSlot(MachineFunction &MF,
Expand Down
20 changes: 15 additions & 5 deletions llvm/lib/Target/AArch64/AArch64MachineFunctionInfo.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,21 @@

using namespace llvm;

static std::optional<uint64_t>
getSVEStackSize(const AArch64FunctionInfo &MFI,
uint64_t (AArch64FunctionInfo::*GetStackSize)() const) {
if (!MFI.hasCalculatedStackSizeSVE())
return std::nullopt;
return (MFI.*GetStackSize)();
}

yaml::AArch64FunctionInfo::AArch64FunctionInfo(
const llvm::AArch64FunctionInfo &MFI)
: HasRedZone(MFI.hasRedZone()),
StackSizeSVE(MFI.hasCalculatedStackSizeSVE()
? std::optional<uint64_t>(MFI.getStackSizeSVE())
: std::nullopt) {}
StackSizeZPR(
getSVEStackSize(MFI, &llvm::AArch64FunctionInfo::getStackSizeZPR)),
StackSizePPR(
getSVEStackSize(MFI, &llvm::AArch64FunctionInfo::getStackSizePPR)) {}

void yaml::AArch64FunctionInfo::mappingImpl(yaml::IO &YamlIO) {
MappingTraits<AArch64FunctionInfo>::mapping(YamlIO, *this);
Expand All @@ -38,8 +47,9 @@ void AArch64FunctionInfo::initializeBaseYamlFields(
const yaml::AArch64FunctionInfo &YamlMFI) {
if (YamlMFI.HasRedZone)
HasRedZone = YamlMFI.HasRedZone;
if (YamlMFI.StackSizeSVE)
setStackSizeSVE(*YamlMFI.StackSizeSVE);
if (YamlMFI.StackSizeZPR || YamlMFI.StackSizePPR)
setStackSizeSVE(YamlMFI.StackSizeZPR.value_or(0),
YamlMFI.StackSizePPR.value_or(0));
}

static std::pair<bool, bool> GetSignReturnAddress(const Function &F) {
Expand Down
63 changes: 37 additions & 26 deletions llvm/lib/Target/AArch64/AArch64MachineFunctionInfo.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,13 +74,10 @@ class AArch64FunctionInfo final : public MachineFunctionInfo {
/// Amount of stack frame size, not including callee-saved registers.
uint64_t LocalStackSize = 0;

/// The start and end frame indices for the SVE callee saves.
int MinSVECSFrameIndex = 0;
int MaxSVECSFrameIndex = 0;

/// Amount of stack frame size used for saving callee-saved registers.
unsigned CalleeSavedStackSize = 0;
unsigned SVECalleeSavedStackSize = 0;
unsigned ZPRCalleeSavedStackSize = 0;
unsigned PPRCalleeSavedStackSize = 0;
bool HasCalleeSavedStackSize = false;
bool HasSVECalleeSavedStackSize = false;

Expand Down Expand Up @@ -137,9 +134,10 @@ class AArch64FunctionInfo final : public MachineFunctionInfo {
/// SVE stack size (for predicates and data vectors) are maintained here
/// rather than in FrameInfo, as the placement and Stack IDs are target
/// specific.
uint64_t StackSizeSVE = 0;
uint64_t StackSizeZPR = 0;
uint64_t StackSizePPR = 0;

/// HasCalculatedStackSizeSVE indicates whether StackSizeSVE is valid.
/// HasCalculatedStackSizeSVE indicates whether StackSizeZPR/PPR is valid.
bool HasCalculatedStackSizeSVE = false;

/// Has a value when it is known whether or not the function uses a
Expand Down Expand Up @@ -300,16 +298,25 @@ class AArch64FunctionInfo final : public MachineFunctionInfo {
TailCallReservedStack = bytes;
}

bool hasCalculatedStackSizeSVE() const { return HasCalculatedStackSizeSVE; }

void setStackSizeSVE(uint64_t S) {
void setStackSizeSVE(uint64_t ZPR, uint64_t PPR) {
StackSizeZPR = ZPR;
StackSizePPR = PPR;
HasCalculatedStackSizeSVE = true;
StackSizeSVE = S;
}

uint64_t getStackSizeSVE() const {
uint64_t getStackSizeZPR() const {
assert(hasCalculatedStackSizeSVE());
return StackSizeSVE;
return StackSizeZPR;
}
uint64_t getStackSizePPR() const {
assert(hasCalculatedStackSizeSVE());
return StackSizePPR;
}

bool hasCalculatedStackSizeSVE() const { return HasCalculatedStackSizeSVE; }

bool hasSVEStackSize() const {
return getStackSizeZPR() > 0 || getStackSizePPR() > 0;
}

bool hasStackFrame() const { return HasStackFrame; }
Expand Down Expand Up @@ -402,23 +409,25 @@ class AArch64FunctionInfo final : public MachineFunctionInfo {
}

// Saves the CalleeSavedStackSize for SVE vectors in 'scalable bytes'
void setSVECalleeSavedStackSize(unsigned Size) {
SVECalleeSavedStackSize = Size;
void setSVECalleeSavedStackSize(unsigned ZPR, unsigned PPR) {
ZPRCalleeSavedStackSize = ZPR;
PPRCalleeSavedStackSize = PPR;
HasSVECalleeSavedStackSize = true;
}
unsigned getSVECalleeSavedStackSize() const {
unsigned getZPRCalleeSavedStackSize() const {
assert(HasSVECalleeSavedStackSize &&
"SVECalleeSavedStackSize has not been calculated");
return SVECalleeSavedStackSize;
"ZPRCalleeSavedStackSize has not been calculated");
return ZPRCalleeSavedStackSize;
}

void setMinMaxSVECSFrameIndex(int Min, int Max) {
MinSVECSFrameIndex = Min;
MaxSVECSFrameIndex = Max;
unsigned getPPRCalleeSavedStackSize() const {
assert(HasSVECalleeSavedStackSize &&
"PPRCalleeSavedStackSize has not been calculated");
return PPRCalleeSavedStackSize;
}

int getMinSVECSFrameIndex() const { return MinSVECSFrameIndex; }
int getMaxSVECSFrameIndex() const { return MaxSVECSFrameIndex; }
unsigned getSVECalleeSavedStackSize() const {
return getZPRCalleeSavedStackSize() + getPPRCalleeSavedStackSize();
}

void incNumLocalDynamicTLSAccesses() { ++NumLocalDynamicTLSAccesses; }
unsigned getNumLocalDynamicTLSAccesses() const {
Expand Down Expand Up @@ -599,7 +608,8 @@ class AArch64FunctionInfo final : public MachineFunctionInfo {
namespace yaml {
struct AArch64FunctionInfo final : public yaml::MachineFunctionInfo {
std::optional<bool> HasRedZone;
std::optional<uint64_t> StackSizeSVE;
std::optional<uint64_t> StackSizeZPR;
std::optional<uint64_t> StackSizePPR;

AArch64FunctionInfo() = default;
AArch64FunctionInfo(const llvm::AArch64FunctionInfo &MFI);
Expand All @@ -611,7 +621,8 @@ struct AArch64FunctionInfo final : public yaml::MachineFunctionInfo {
template <> struct MappingTraits<AArch64FunctionInfo> {
static void mapping(IO &YamlIO, AArch64FunctionInfo &MFI) {
YamlIO.mapOptional("hasRedZone", MFI.HasRedZone);
YamlIO.mapOptional("stackSizeSVE", MFI.StackSizeSVE);
YamlIO.mapOptional("stackSizeZPR", MFI.StackSizeZPR);
YamlIO.mapOptional("stackSizePPR", MFI.StackSizePPR);
}
};

Expand Down
4 changes: 2 additions & 2 deletions llvm/lib/Target/AArch64/AArch64RegisterInfo.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -644,7 +644,7 @@ bool AArch64RegisterInfo::hasBasePointer(const MachineFunction &MF) const {
if (ST.hasSVE() || ST.isStreaming()) {
// Frames that have variable sized objects and scalable SVE objects,
// should always use a basepointer.
if (!AFI->hasCalculatedStackSizeSVE() || AFI->getStackSizeSVE())
if (!AFI->hasCalculatedStackSizeSVE() || AFI->hasSVEStackSize())
return true;
}

Expand Down Expand Up @@ -784,7 +784,7 @@ AArch64RegisterInfo::useFPForScavengingIndex(const MachineFunction &MF) const {
assert((!MF.getSubtarget<AArch64Subtarget>().hasSVE() ||
AFI->hasCalculatedStackSizeSVE()) &&
"Expected SVE area to be calculated by this point");
return TFI.hasFP(MF) && !hasStackRealignment(MF) && !AFI->getStackSizeSVE() &&
return TFI.hasFP(MF) && !hasStackRealignment(MF) && !AFI->hasSVEStackSize() &&
!AFI->hasStackHazardSlotIndex();
}

Expand Down
3 changes: 2 additions & 1 deletion llvm/test/DebugInfo/AArch64/asan-stack-vars.mir
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -366,7 +366,8 @@ frameInfo:
maxCallFrameSize: 0
localFrameSize: 144
machineFunctionInfo:
stackSizeSVE: 0
stackSizeZPR: 0
stackSizePPR: 0
stack:
- { id: 0, name: StackGuardSlot, offset: -40, size: 8, alignment: 8,
stack-id: default, local-offset: -8 }
Expand Down
3 changes: 2 additions & 1 deletion llvm/test/DebugInfo/AArch64/compiler-gen-bbs-livedebugvalues.mir
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,8 @@ frameInfo:
hasCalls: true
maxCallFrameSize: 0
machineFunctionInfo:
stackSizeSVE: 0
stackSizeZPR: 0
stackSizePPR: 0
stack:
- { id: 0, type: spill-slot, offset: -20, size: 4, alignment: 4, stack-id: default }
- { id: 1, type: spill-slot, offset: -8, size: 8, alignment: 8, stack-id: default,
Expand Down