Using App.config with SpecFlowCompatibility plugin does not work

[gasparnagy]

Reqnroll Version

3.3.2

Which test runner are you using?

NUnit

Test Runner Version Number

4.0.1

.NET Implementation

.NET Framework 4.6.2 or later

Test Execution Method

Visual Studio Test Explorer

Content of reqnroll.json configuration file

Issue Description

The SpecFlowCompatibilityProject.Net472 project in https://github.com/reqnroll/Reqnroll.ExploratoryTestProjects/ builds, but the tests fail with:

 Message: 
OneTimeSetUp: System.Configuration.ConfigurationErrorsException : An error occurred creating the configuration section handler for specFlow: Could not load type 'Reqnroll.SpecFlowCompatibility.ReqnrollPlugin.ConfigurationSectionHandler' from assembly 'Reqnroll.SpecFlowCompatibility.ReqnrollPlugin'. (W:\Reqnroll\Reqnroll.ExploratoryTestProjects\SpecFlowCompatibilityProject\SpecFlowCompatibilityProject.Net472\bin\Debug\net472\SpecFlowCompatibilityProject.Net472.dll.config line 4)
  ----> System.TypeLoadException : Could not load type 'Reqnroll.SpecFlowCompatibility.ReqnrollPlugin.ConfigurationSectionHandler' from assembly 'Reqnroll.SpecFlowCompatibility.ReqnrollPlugin'.

  Stack Trace: 
BaseConfigurationRecord.FindAndEnsureFactoryRecord(String configKey, Boolean& isRootDeclaredHere)
BaseConfigurationRecord.GetSectionRecursive(String configKey, Boolean getLkg, Boolean checkPermission, Boolean getRuntimeObject, Boolean requestIsHere, Object& result, Object& resultRuntimeObject)
BaseConfigurationRecord.GetSection(String configKey)
ConfigurationManager.GetSection(String sectionName)
AppConfigConfigurationLoader.get_HasAppConfig()
<>c.<Initialize>b__0_0(Object _, ConfigurationDefaultsEventArgs args)
RuntimePluginEvents.RaiseConfigurationDefaults(ReqnrollConfiguration reqnrollConfiguration)
ContainerBuilder.CreateGlobalContainer(Assembly testAssembly, IRuntimeConfigurationProvider configurationProvider)
TestRunnerManager.CreateTestRunnerManager(Assembly testAssembly, IContainerBuilder containerBuilder)
<>c__DisplayClass47_0.<GetTestRunnerManager>b__0(Assembly assembly)
ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
TestRunnerManager.GetTestRunnerManager(Assembly testAssembly, IContainerBuilder containerBuilder, Boolean createIfMissing)
<OnTestRunStartAsync>d__51.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
ExceptionDispatchInfo.Throw()
TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
TaskAwaiter.GetResult()
<AssemblyInitializeAsync>d__0.MoveNext() line 16
--- End of stack trace from previous location where exception was thrown ---
ExceptionDispatchInfo.Throw()
TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
GenericAdapter`1.GetResult()
AsyncToSyncAdapter.Await[TResult](Func`1 invoke)
SetUpTearDownItem.RunSetUpOrTearDownMethod(TestExecutionContext context, IMethodInfo method)
SetUpTearDownItem.RunSetUp(TestExecutionContext context)
BeforeTestCommand.Execute(TestExecutionContext context)
CompositeWorkItem.PerformOneTimeSetUp()
--TypeLoadException
TypeUtil.GetTypeWithReflectionPermission(IInternalConfigHost host, String typeString, Boolean throwOnError)
RuntimeConfigurationFactory.Init(RuntimeConfigurationRecord configRecord, FactoryRecord factoryRecord)
RuntimeConfigurationFactory.InitWithRestrictedPermissions(RuntimeConfigurationRecord configRecord, FactoryRecord factoryRecord)
RuntimeConfigurationRecord.CreateSectionFactory(FactoryRecord factoryRecord)
BaseConfigurationRecord.FindAndEnsureFactoryRecord(String configKey, Boolean& isRootDeclaredHere)

If App.config is replaced with reqnroll.json, it works again.

Checked with v2.4.1 and it was not working either, so the problem does not seem to be new.

Steps to Reproduce

• https://github.com/reqnroll/Reqnroll.ExploratoryTestProjects/
• open SpecFlowCompatibilityProject\SpecFlowCompatibilityProject.slnx
• build
• run tests

Link to Repro Project

https://github.com/reqnroll/Reqnroll.ExploratoryTestProjects/

[gasparnagy]

We have to decide if we want to fix this at all.

[clrudolphi]

And lack of any bug reports on this for over a year would imply that no one is using app.config with SpecFlowCompatibility. What does telemetry say about usage rate of SpecFlow Compatibility over all? Are we at the point of deprecating it completely?

[gasparnagy]

And lack of any bug reports on this for over a year would imply that no one is using app.config with SpecFlowCompatibility. What does telemetry say about usage rate of SpecFlow Compatibility over all? Are we at the point of deprecating it completely?

Unfortunately we don't have telemetry about this. But I agree with the deprecation.

I have found this issue while checking the VS extension discovery on all of our test projects. I'm fine closing this as not planned, just wanted to record the problem.

[304NotModified]

As reqnroll.json works and that is in sync with other frameworks, I would vote on won't fix and removal of the app.config code/config/docs etc.