opensearch environment variables

[Scarehcrow]

I saw inhere: #1671
that there is support in the environment variables for opensearch user/password combination.
But it's for me unclear what I should use as name.

This topic #2161 speaks about minio, which should be the same way of working:
You need to specify credentials using this syntax as env var:
_set MinioCredentials=accesskey:xxxxxx;secretkey:yyyyyyyyyy

Or as a java sys prop:
iped.exe -DMinioCredentials=accesskey:xxxxxx;secretkey:yyyyyyyyyy -d input -o output

Or as an IPED extra property:
iped.exe -XMinioCredentials=accesskey:xxxxxx;secretkey:yyyyyyyyyy -d input -o output_

But what's the exact way of doing it for opensearch/elasticsearch? Might be better off adding it to either the conf file or the documentation somewhere.

Thanks in advance for any info you can provide

[lfcnassif]

@hauck-jvsh could you help him?

[lfcnassif]

Seems @hauck-jvsh is busy, so I'll try to answer. Looking at the code here:
https://github.com/sepinf-inc/IPED/blob/master/iped-engine/src/main/java/iped/engine/task/index/ElasticSearchIndexTask.java

There are these options:

Passing as env var:
set elastic=user:xxxx;password:yyyyy;indexName:zzzzzz

Or as a java sys prop:
iped.exe -Delastic=user:xxxx;password:yyyyy;indexName:zzzzzz -d input -o output

Or as an IPED extra property:
iped.exe -Xelastic=user:xxxx;password:yyyyy;indexName:zzzzzz -d input -o output

Hope it helps

[lfcnassif]

For sure, it should be documented.

[Scarehcrow]

I've managed to get this working, thanks for the info!

[lfcnassif]

I'm converting this to discussion.

[hauck-jvsh]

Hi, usually I pass the credentials as a command line parameter like this -Xelastic="indexName:nameoftheindex;user:username;password:password".

[reacan]

Hey there!

I am wondering which versions of ElasticSearch are supported?

I am trying to push from IPED 4.1.2 to ElasticSearch 8.17.3 and I get some sort of an unknown setting error.

Suppressed: org.opensearch.client.ResponseException: method [PUT], host [https://localhost:9200], URI [/ipedtest?master_timeout=30s&timeout=30s], status line [HTTP/1.1 400 Bad Request]
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"unknown setting [index.knn] please check that any required plugins are installed, or check the breaking changes documentation for removed settings"}],"type":"illegal_argument_exception","reason":"unknown setting [index.knn] please check that any required plugins are installed, or check the breaking changes documentation for removed settings"},"status":400}

Or is there a plugin that I should install for the ElasticSearch?

[lfcnassif]

Hi @reacan,

We moved to OpenSearch due to ElasticSearch license changes. Currently we are using OpenSearch 2.1 java plugin, I think it is compatible to old ElasticSearch versions, maybe @hauck-jvsh can clarify.

[reacan]

Hi @lfcnassif

Thank you! It works with OpenSearch 2.12.0!

[lfcnassif]

According to @hauck-jvsh, it should work up to ElasticSearch 7.x, later versions diverged from OpenSearch.

[paulobreim]

I use elasticsearch-9.0.2 and yesterday I was testing IPED and it really has this OpenSearch issue, which I don't have installed. I tried turning off similarity items but it didn't work.
I would like to know if I need to use an earlier version of Elasticsearch or if there are any changes to use it with the most current versions of Elasticsearch.

My purpose in generating an output in Elasticsearch would be to see the predictions for developing a mobile application that can perform queries in those cases.

tks
@paulobreim