summaryrefslogtreecommitdiff
path: root/src/amd/compiler/aco_assembler.cpp
Commit message (Collapse)AuthorAgeFilesLines
* aco/assembler: support VOP3P with DPPGeorg Lehmann2023-05-121-43/+39
| | | | | Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22698>
* aco: allow no export instruction for gfx10+ fsQiang Yu2023-04-281-1/+4
| | | | | | | | | radeonsi will generate no export instruction when no discard and no output. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Signed-off-by: Qiang Yu <yuq825@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22727>
* aco,radv: support symbol relocation in acoQiang Yu2023-04-281-3/+25
| | | | | | Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Signed-off-by: Qiang Yu <yuq825@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22727>
* aco/assembler: support VOP12C opselGeorg Lehmann2023-03-301-2/+17
| | | | | Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22069>
* aco: Don't include headers from radv.Timur Kristóf2023-03-081-0/+1
| | | | | | Signed-off-by: Timur Kristóf <timur.kristof@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21696>
* aco/assembler/gfx11: simplify 16bit VOP12C promotion to VOP3Georg Lehmann2023-03-071-76/+37
| | | | | | | With the shared struct for modifies, this is can be a lot cleaner now. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21761>
* aco: remove VOP[123C]P? structsGeorg Lehmann2023-03-071-2/+2
| | | | | | Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21023>
* aco: Handle NSA with vectors in get_mimg_nsa_dwords.Georg Lehmann2023-01-111-1/+2
| | | | | | | No Foz-DB changes. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20370>
* aco/gfx11: allow true 16-bit instructions to access v128+Rhys Perry2023-01-101-21/+93
| | | | | | | | | | | It looks like the LLVM assembler promotes true 16-bit instructions to VOP3 in this case. No fossil-db changes. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20251>
* aco: align scratch size during assemblyRhys Perry2023-01-091-0/+3
| | | | | | | | | This lets us use less scratch if both VGPR spilling and scratch intrinsics are used. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20534>
* aco/gfx11: update s_code_end paddingRhys Perry2023-01-061-1/+1
| | | | | | | | | Match ac_rtld_open(). Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Cc: 22.3 <mesa-stable> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20536>
* aco/gfx11: fix s_waitcnt printingRhys Perry2022-09-301-1/+1
| | | | | | Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17710>
* aco: add VINTERP instruction formatRhys Perry2022-09-261-0/+18
| | | | | | Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17333>
* aco: add LDSDIR instruction formatRhys Perry2022-09-261-0/+11
| | | | | | Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17333>
* aco: update assembler for GFX11Rhys Perry2022-09-261-45/+115
| | | | | | Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17333>
* aco: add GFX11 opcode numbersRhys Perry2022-09-261-1/+3
| | | | | | Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17333>
* aco: fix assembly of MUBUF-to-LDS loadsRhys Perry2022-09-261-2/+2
| | | | | | | | These have an implicit m0 read and don't write VGPRs. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17333>
* aco: add reg() helper to assemblerRhys Perry2022-09-261-82/+99
| | | | | | | | SGPR encoding is slightly different on GFX11. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17333>
* aco: rename Interp_instruction to VINTRP_instructionRhys Perry2022-09-261-1/+1
| | | | | | | | These is clearer since GFX11 adds another interpolation format. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17333>
* aco: fix long-jump version of discard early exitRhys Perry2022-08-251-6/+14
| | | | | | | | | | | It isn't safe to modify the exec mask before the discard block, and the definition interferes with GFX11 NOP insertion. Just use s[0:1] instead, since we won't be using it. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18125>
* aco: fix assembly of vopc_sdwa writing execRhys Perry2022-08-161-1/+2
| | | | | | | | | | We would assemble an instruction writing vcc instead. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Fixes: 5ffc73896f3 ("aco/assembler: Fix v_cmpx with SDWA.") Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18077>
* aco/assembler: Fix v_cmpx pre GFX10.Georg Lehmann2022-08-151-1/+6
| | | | | | | | The second destination is implicit exec. Signed-off-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18049>
* aco/assembler: Fix v_cmpx with SDWA.Georg Lehmann2022-08-151-1/+1
| | | | | | | | | We need to use the implicit destination. Fixes: baab6f18c91 ("aco: Optimize branching sequence during SSA elimination.") Signed-off-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18049>
* aco: prevent adding DONE/VM to the last export if the FS has an epilogSamuel Pitoiset2022-07-181-2/+4
| | | | | | | | | If the fragment shader exports MRTZ and the epilog some color exports, DONE/VM should be added to the last export. Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17485>
* aco: do not abort if the FS doesn't export anything but has an epilogSamuel Pitoiset2022-07-181-1/+7
| | | | | | | | | The main fragment shader can only export MRTZ (if present) and the epilog will export colors. Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17485>
* aco: improve support for scratch_* instructionsRhys Perry2022-07-081-1/+5
| | | | | | Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17079>
* aco: make FLAT_instruction::offset signedRhys Perry2022-07-081-4/+7
| | | | | | Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17079>
* aco/assembler: Fix s_bitreplicate_b64_b32 on GFX9.Georg Lehmann2022-07-081-4/+0
| | | | | | | | This seems to be a relic from before aco added per generation opcodes. Signed-off-by: Georg Lehmann <dadschoorse@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/17405>
* aco: don't use 32-bit fp inline constants for fp16 vop3p literalsRhys Perry2022-07-051-3/+3
| | | | | | | | | | If we're applying the literal 0x3f800000 to a fp16 vop3p instruction, we shouldn't use the 1.0 inline constant, because the hardware will use the 16-bit 1.0: 0x00003c00. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16296>
* aco: disable sdwa on gfx11Rhys Perry2022-05-311-0/+1
| | | | | | | | | | | | | Instead of SDWA v_mov_b32/v_xor_b32, we can use a combination of v_add_u16/v_sub_u16 (add/sub swap, similar to xor swap) and v_perm_b32 with a literal. I don't know yet if GFX11 adds any new instructions which makes this easier, but this approach should have full functionality. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16595>
* aco: only add/subtract low bits of program addressesRhys Perry2022-05-231-5/+2
| | | | | | | | | | | | | fossil-db (Sienna Cichlid): Totals from 4007 (2.47% of 162293) affected shaders: Instrs: 3733239 -> 3728018 (-0.14%) CodeSize: 20770340 -> 20749456 (-0.10%) Latency: 46883958 -> 46872764 (-0.02%); split: -0.02%, +0.00% InvThroughput: 10550392 -> 10548698 (-0.02%); split: -0.02%, +0.00% Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16460>
* aco: fix p_constaddr with a non-zero offsetRhys Perry2022-05-231-2/+4
| | | | | | | | | Seems this broke a while ago and we never noticed. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Fixes: 0af7ff49fde ("aco: lower p_constaddr into separate instructions earlier") Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16460>
* amd: change chip_class naming to "enum amd_gfx_level gfx_level"Marek Olšák2022-05-131-53/+53
| | | | | | | | This aligns the naming with PAL. Acked-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> Acked-by: Pierre-Eric Pellou-Prayer <pierre-eric.pelloux-prayer@amd.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/16469>
* aco: don't encode src2 for v_writelane_b32_e64Rhys Perry2022-02-031-0/+4
| | | | | | | | | | | | | | Encoding src2 doesn't cause issues for print_asm() because we have a workaround there, but it does for RGP and it seems the developers are not interested in fixing it. https://github.com/GPUOpen-Tools/radeon_gpu_profiler/issues/61 Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Tested-by: Tatsuyuki Ishi <ishitatsuyuki@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Cc: mesa-stable Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14832>
* aco: support DPP8Tatsuyuki Ishi2021-12-311-3/+17
| | | | | Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13971>
* aco: remove explicit dst_preserve flagDaniel Schürmann2021-09-021-1/+1
| | | | | | | | | Instead, we can rely on the fact that subdword definitions must preserve the unused bits while dword definitions either pad or sign-extend. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12640>
* aco: rewrite SDWA selectorDaniel Schürmann2021-09-021-20/+7
| | | | | | | | | | | | | | This commit introduces a new struct SubdwordSel in order to ease and clean up the usage of SDWA selections. This includes removing the distinction between register-allocated and fixed SDWA selections. Instead, SDWA selections can now also access the high bits of subdword variables. Alignment and sizes are validated accordingly. Size, offset and sign_extend can be evaluated via helper methods. Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12640>
* aco: Remove use of deprecated Operand constructorsTony Wasserka2021-07-131-7/+7
| | | | | | | | | | | | | | | | | This migration was done with libclang-based automatic tooling, which performed these replacements: * Operand(uint8_t) -> Operand::c8 * Operand(uint16_t) -> Operand::c16 * Operand(uint32_t, false) -> Operand::c32 * Operand(uint32_t, bool) -> Operand::c32_or_c64 * Operand(uint64_t) -> Operand::c64 * Operand(0) -> Operand::zero(num_bytes) Casts that were previously used for constructor selection have automatically been removed (e.g. Operand((uint16_t)1) -> Operand::c16(1)). Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11653>
* aco: Format.Daniel Schürmann2021-07-121-84/+107
| | | | | | | Manually adjusted some comments for more intuitive line breaks. Reviewed-by: Tony Wasserka <tony.wasserka@gmx.de> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11258>
* aco: add 'common/' and 'llvm/' prefix to #includesDaniel Schürmann2021-07-121-1/+2
| | | | | Reviewed-by: Tony Wasserka <tony.wasserka@gmx.de> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11271>
* aco: add missing Licenses and remove Authors from filesDaniel Schürmann2021-07-121-1/+23
| | | | | Reviewed-by: Tony Wasserka <tony.wasserka@gmx.de> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11271>
* aco: reorder and cleanup #includesDaniel Schürmann2021-07-121-7/+8
| | | | | Reviewed-by: Tony Wasserka <tony.wasserka@gmx.de> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11271>
* aco: fix emitting a16 for MIMG instructions on GFX10+Samuel Pitoiset2021-07-071-1/+1
| | | | | | Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11734>
* aco: fix emitting d16 for MIMG instructions on GFX9+Samuel Pitoiset2021-07-071-1/+1
| | | | | | Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> Reviewed-by: Rhys Perry <pendingchaos02@gmail.com> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11734>
* aco: fix emitting literal offsets with SMEM on GFX7Samuel Pitoiset2021-06-091-2/+4
| | | | | | | | | | | | | When the offset is negative, reg() isn't 255. Fix this by splitting SGPR and literal emission. While we are at it, adjust a comment saying that literals are also accepted on GFX6 which is wrong. Fixes another batch of robustness tests. Cc: 21.1 mesa-stable Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11247>
* aco: Implement bvh64_intersect_ray_amd intrinsic.Bas Nieuwenhuizen2021-05-181-1/+2
| | | | | Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10818>
* aco: fix NSA MIMG followed by MUBUF/MTBUFRhys Perry2021-03-171-9/+12
| | | | | | | | | No fossil-db changes on GFX10. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Timur Kristóf <timur.kristof@gmail.com> Fixes: c353895c922 ("aco: use non-sequential addressing") Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9187>
* aco: lower p_constaddr into separate instructions earlierRhys Perry2021-03-111-42/+28
| | | | | | | | | This allows them to be scheduled properly and simplifies the assembler a little. Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8994>
* aco: return references in instruction cast methodsRhys Perry2021-01-221-130/+130
| | | | | | Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8595>
* aco: use format-check methodsRhys Perry2021-01-221-11/+11
| | | | | | Signed-off-by: Rhys Perry <pendingchaos02@gmail.com> Reviewed-by: Daniel Schürmann <daniel@schuermann.dev> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8595>
View Lorry Controller Status