Here is my shader code for setting alpha blending on. It is not working at all for some reason. It worked under fx_4_0 but doesn't work under fx_5_0.

BlendState EnableBlend
{
    BlendEnable[0] = TRUE;
    SrcBlendAlpha = 5;
    DestBlendAlpha = 7;
};

technique11 GUI
{
    pass Render
    {
        SetVertexShader( CompileShader( vs_4_0, VS() ) );
        SetGeometryShader( NULL );
        SetPixelShader( CompileShader( ps_4_0, PS() ) );
                
        SetRasterizerState(DisableCulling);       
        SetDepthStencilState(DepthEnabling, 0);
        SetBlendState(EnableBlend, float4( 0.0f, 0.0f, 0.0f, 0.0f ), 0xFFFFFFFF);
    }
}

As far as I know the August 2009 DX SDK comes with support for the new fx_5_0 target so if you are using the March SDK there still was no fx_5_0 target -  other than that I can't help you I am afraid.

I presume you're using the new Effects 11 runtime? Do you have the debug layer on? Does it say anything? There is a bug running on feature levels < 11 due to the fact that it always passes a class linkage object to create shader. Since you have the source code, you should be able to work around it until the next release...

else
        {
            // This is a regular shader
            if( pShader->pReflectionData->RasterizedStream == D3D11_SO_NO_RASTERIZED_STREAM )
                pShader->IsValid = FALSE;
            else if( FAILED( (m_pDevice->*(pShader->pVT->pCreateShader))( (UINT *) pShader->pReflectionData->pBytecode, pShader->pReflectionData->BytecodeLength, (m_pDevice->GetFeatureLevel() == D3D_FEATURE_LEVEL::D3D_FEATURE_LEVEL_11_0)?m_pClassLinkage:NULL, &pShader->pD3DObject) ) )
                pShader->IsValid = FALSE;
        }

yes I fixed that part like so, I have everything working but alpha blending. I can't seem to figure out what settings to do. It is almost as though the effect framework ignores it totally.

Does the disassembler disassemble your effect for you? I don't know if we did that work yet. You could try stepping into the blend state creation code and see if it thinks there are any blend state objects at around line 1208 of EffectNonRuntime.cpp

I have verified that it is working so I am assuming my issue is that I was getting away with sloppy settings under 10 that is now fixed in 11. The effect layer is setting the blend state correctly so I just must not have the right state.

BlendState EnableBlend
{
    BlendEnable[0] = TRUE;
    SrcBlend[0] = 3;
    DestBlend[0] = 9;
    SrcBlendAlpha[0] = 5;
    DestBlendAlpha[0] = 6;
};

this seems to work for the most part but source alpha = 0 is not 100% transparent or  just not being skipped for some reason. Any ideas?

These are all of the states as defined in Binary\EffectStates11.h, try initializing all of them, I'm guessing perhaps the blend op isn't what you're expecting? Take a look at the definition of the g_rv* values to see what is valid, for example for BLENDOP, look at g_rvBLENDOP.

    { "AlphaToCoverageEnable",      EBT_Blend,          D3D10_SVT_BOOL,               1, 1, FALSE, g_rvBOOL,                ELHS_AlphaToCoverage,           offsetof_fx(SBlendBlock, BackingStore.AlphaToCoverageEnable),                  0 },
    { "BlendEnable",                EBT_Blend,          D3D10_SVT_BOOL,               1, 8, FALSE, g_rvBOOL,                ELHS_BlendEnable,               offsetof_fx(SBlendBlock, BackingStore.RenderTarget[0].BlendEnable),            strideof(SBlendBlock, BackingStore.RenderTarget) },
    { "SrcBlend",                   EBT_Blend,          D3D10_SVT_UINT,               1, 8, TRUE,  g_rvBLEND,               ELHS_SrcBlend,                  offsetof_fx(SBlendBlock, BackingStore.RenderTarget[0].SrcBlend),               strideof(SBlendBlock, BackingStore.RenderTarget) },
    { "DestBlend",                  EBT_Blend,          D3D10_SVT_UINT,               1, 8, TRUE,  g_rvBLEND,               ELHS_DestBlend,                 offsetof_fx(SBlendBlock, BackingStore.RenderTarget[0].DestBlend),              strideof(SBlendBlock, BackingStore.RenderTarget) },
    { "BlendOp",                    EBT_Blend,          D3D10_SVT_UINT,               1, 8, TRUE,  g_rvBLENDOP,             ELHS_BlendOp,                   offsetof_fx(SBlendBlock, BackingStore.RenderTarget[0].BlendOp),                strideof(SBlendBlock, BackingStore.RenderTarget) },
    { "SrcBlendAlpha",              EBT_Blend,          D3D10_SVT_UINT,               1, 8, TRUE,  g_rvBLEND,               ELHS_SrcBlendAlpha,             offsetof_fx(SBlendBlock, BackingStore.RenderTarget[0].SrcBlendAlpha),          strideof(SBlendBlock, BackingStore.RenderTarget) },
    { "DestBlendAlpha",             EBT_Blend,          D3D10_SVT_UINT,               1, 8, TRUE,  g_rvBLEND,               ELHS_DestBlendAlpha,            offsetof_fx(SBlendBlock, BackingStore.RenderTarget[0].DestBlendAlpha),         strideof(SBlendBlock, BackingStore.RenderTarget) },
    { "BlendOpAlpha",               EBT_Blend,          D3D10_SVT_UINT,               1, 8, TRUE,  g_rvBLENDOP,             ELHS_BlendOpAlpha,              offsetof_fx(SBlendBlock, BackingStore.RenderTarget[0].BlendOpAlpha),           strideof(SBlendBlock, BackingStore.RenderTarget) },
    { "RenderTargetWriteMask",      EBT_Blend,          D3D10_SVT_UINT8,              1, 8, FALSE, NULL,                    ELHS_RenderTargetWriteMask,     offsetof_fx(SBlendBlock, BackingStore.RenderTarget[0].RenderTargetWriteMask),  strideof(SBlendBlock, BackingStore.RenderTarget) },

I have set the ops explicitely to add. Still no dice as a side note it doesn't matter what I set the SrcBlendAlpha and DestBlendAlpha to I get the same results. I even set them to 2 which is zero and still I get a correct blend but 0 is not completely transparent. I am assuming this is an issue with vista or nvidia's 8600M GT driver.

BlendState EnableBlend
{
    BlendEnable[0] = TRUE;
    SrcBlend = 5;
    DestBlend = 6;
};

ok found the issue this is the fix. I was just confused as to what the docs meant.