Do you think it would be worth it to check the sizes match?
We can probably do it with get_element_type() and get_num_units()?

Might be worth checking. Will add now.

I guess it doesn't matter much since there's a check for pointers in the assert in the other file, but is this the expected behavior to check if the pointee types are equal since we can bitcast pointers of different types?

The code path calling this funcition in the assert should not be hit by pointers at all - it is meant to bitcast vectors.

The idea behind known_eq is to serve as a saner replacement for the current Eq implementation of Type.

As we discussed before, comparing Types / Rvalues on the libgccjit side is quite hard to do, and would require a lot of code to be added.

Here, I am seeing if I could possibly work around that issue, by doing some of that comparison logic on the Rust side.

Really, I am trying to see if I could implement the functions we require entirely on the Rust side, subverting the limitations of the GCC API.

One thing I am considering attempting is field-vise comparison of structs(so that structs with identical elements compare as equal).

This could be quite useful for our constant deduplication code. Since my last attempt at fixing that leak introduces non-determinism to the compiler, we will have to revert and replace that.

At one point, I suggested we compare the types of Rvalues before comparing them using string formatting.

I was told that will not work because we can't reliably compare types. known_eq solves that problem(once I get struct comparisons working). If 2 rvalues have different types, we know they can't be equal, so we don't do the string-based comparison.

I belive this would remove most of that memory leak in a sound and reliable way. I have other ways to aleviate that issue, but this will be the most impactfull.

I still believe that using that using global_set_initializer() would be better than this approach. Was there any reason it would not work?

Also, I would be very wary of a struct comparison done like that: we need to take some attributes into account like packed and possibly some others: I much prefer to let that job to the gcc backend even though I'm not sure right now how that could work since it would involve the target part of the compiler.

global_set_initializer(and globlal_set_initialized_rvalue) required globals to have unique names, and I think that messed with the string-based comparisons. I also belive could not get it to ever run right - global_set_initializer always errored out when I used it. I belive that was caused by some kind of type issue - it always complained about the type of the global and the type of the initializer rvalue being different. Maybe this is related to us calling get_aligned needlessly.

Also, I would be very wary of a struct comparison done like that: we need to take some attributes into account like packed and possibly some others: I much prefer to let that job to the gcc backend even though I'm not sure right now how that could work since it would involve the target part of the compiler.

Which is why known_eq will not return Some(true) for structs(at least for now). For structs, the current version only allows us to check that structs are unequal which is fine for our purposes.

Overall, the idea is for known_eq to return Some(true) if and only if the types are guaranteed to be equal, and return Some(false) if we can show that they are not equal.

For the global de-duplication, this comparison is simply an optimization. All it does skip the more costly comparison if we know for sure it will return false.

Additionally, for our purposes, catching most of the cases where types are not equal is already good enough.

If a sanity check can catch 99.9% of issues early, then it is worth it in my book.

Also global_set_initializer will not help us with deduplication, at all. We will still have to manage the lvalues by ourselves, and deduplicate them ourselves.

Also global_set_initializer will not help us with deduplication, at all.

Oh, sorry: I was mixing things up with the reduction of the memory usage.

global_set_initializer(and globlal_set_initialized_rvalue) required globals to have unique names, and I think that messed with the string-based comparisons. I also belive could not get it to ever run right - global_set_initializer always errored out when I used it. I belive that was caused by some kind of type issue - it always complained about the type of the global and the type of the initializer rvalue being different. Maybe this is related to us calling get_aligned needlessly.

Please tell me if a fix to libgccjit related to global_set_initializer would help you with your fix the reduce the memory usage (e.g. remove the non-determinism). I could take a look.