The shader creation helpers returned S_FALSE when compilation produced no
bytecode - but S_FALSE is a success code, so the FAILED() checks in the
D3D11VertexShader/D3D11FragmentShader constructors never fired and failed_ was
never set. Return E_FAIL instead, and only hand out the bytecode when the shader
object was actually created.
Failed() had no callers at all, so a failed shader was handed to the draw as
usual: VSSetShader(nullptr), CreateInputLayout on empty bytecode, and the
ignored HRESULT from SetupDecFmtForDraw meant IASetInputLayout(nullptr). The
draw then did nothing, with no log line to explain it. Skip the draw and warn
instead, like the GL backend does.
The cleanup loop was hardcoded to 12 entries while the array holds more and
levels can exceed that with a texture replacement pack - a 8192-pixel
replacement has 14 mip levels, so every level from 12 up leaked on each texture
build. Use ARRAY_SIZE, and shrink the array to 16 since D3D11 caps textures at
16384 pixels anyway.
The out-of-memory bail-out returned before that loop, leaking every level
decoded so far. Since the entry ends up without a texture it gets rebuilt, and
leaks again, every following frame.
Nothing has generated or used a geometry shader since the GS paths were removed
- GeometryShaderGenerator is gone, and ShaderWriter's BeginGSMain/EndGSMain had
no callers at all. Removes ShaderStage::Geometry and everything hanging off it:
the GS preambles and GSMain helpers in ShaderWriter, the stage mappings in all
three thin3d backends, the D3D11 geometry shader plumbing (curGS_, the pipeline
and module members, gs_4_0 compilation), CreateGeometryShaderD3D11, the unused
PipelineFlags::USES_GEOMETRY_SHADER and PipelineManagerVulkan's
UsesGeometryShader().
Also stop enabling the Vulkan geometryShader device feature, since we no longer
have any use for it.
Kept on purpose: the device feature is still listed in the feature dumps (like
other capabilities we don't use), and the Vulkan shader cache header keeps its
now-always-zero geometry shader count so the on-disk format stays compatible.
Previously, "Smart 2D texture filtering" was limited to through-mode
draws, but many games use fully transformed draws to do 2D elements.
An example of this is Outrun in the menus, and it's always been plagued by
horrible filtering artifacts.
With the improved Smart 2D texture filtering, the artifacts are gone!
Of course, instead of glitchy bilinear you instead get point sampled
texturing so it looks more pixellated, but no more ugly borders between letters
in the game.
Henrik Rydgård