dkd Blog

Blog

Was wir machen. Wie wir denken. Wofür wir brennen.

Latest Apache Solr for TYPO3 releases

solrTYPO3releaseapache solrextsolr
Timo Hund

In february we’ve released several updates of our Apache Solr extensions. In this blog post, I want to summarize these releases and the highlights.

EXT:solr 9

With EXT:solr 9 you can build a fast and flexible search for your TYPO3 website. You can index „pages“ and any other kind of TCA records with a flexible indexing configuration that is done with TypoScript.

 

The highlights of EXT:solr 9 are:
 

Solarium PHP-API:
Since the old PHP-API solrphpclient is not maintained anymore and there are better alternatives available, we’ve switched to the Solarium PHP-API. This API is used by many PHP projects and used by many users (4.3 million downloads)


Apache Solr 7.6 support:
With EXT:solr 9 we’ve also shipped a newer Version of Apache Solr. Please check the Release Notes of Apache Solr 7.6 to get an overview of the new features.

TYPO3 9 LTS:
EXT:solr 9 can be installed with TYPO3 9 LTS. Many changes (e.g. the migration from pages_language_overlay to pages table) are reflected in EXT:solr 9 but not all features of TYPO3 9 are fully supported. Especially when you use the „site handling“ you should check if your setup is supported. Depending on the financing, we will provide a full site handling support later in 2019


Read and Write connections:
Up to now each site had one Solr connection for reading and writing. In most of the cases this was good enough when you want to index and search in the same core. Some setups require a more flexible approach:

  • E.g. when you want to clean a core and re-index that data your index is not complete for some time on the live site
  • When you want to do a master/slave setup for performance reasons(e.g. by using a slave node on the web server) this was not possible by now 

With a separation of read and write connections, this is now possible. With these building blocks, you could e.g.

  • Index into a shadow core (that is the write core) and swap read and write core when your re-index is done
  • Install a slave server on your frontend server and index into a dedicated master node that act's as a Solr master server

Besides these highlights, a lot of other small changes and bugfixes have been provided with EXT:solr 9.0.0. You can find a complete list of them in the release notes.

As a followup, we've also provided a first maintenance release 9.0.1.

EXT:tika 4

If you want to extract the content and metadata of files in TYPO3 you can use EXT:tika to do that.

EXT:tika uses Apache Tika, as Tika Sever, Tika App or from Solr to retrieve this data from your documents.

EXT:tika 4 contains the following changes:

  • TYPO3 9 and EXT:solr 9 compatibility
  • Support of version 1.20 of Apache Tika
  • Mpeg/audio files can now be extracted with Solr Cell.


Beside that EXT:tika 4 contains a few small bugfixes. To get an overview of all changes please read the release notes.

EXT:solrfal 6

If you want to index your files in TYPO3 from FAL, you can use our addon EXT:solrfal. In combination with EXT:tika, you are able to index the file contents into your Apache Solr search.

With EXT:solrfal 6 we've provided a compatibility release for EXT:solr 9.

EXT:solrfal 6 does not contain new features, but provides several changes that have been required for EXT:solr 9, TYPO3 9 and EXT:solrconsole 2.

EXT:solrfal is available for our EB partners on http://www.typo3-solr.com

EXT:solrfluidgrouping 2.0.0

EXT:solrfluidgrouping only contains compatibility changes for the Solarium PHP-API and small changes.

Please read the release notes for more information.

EXT:solrconsole 2

In 2018 we've provided a first version of the solrconsole addon. This extension provides several commands on the command line that allows you to automate recurring tasks.

The highlights of solrconsole 2 are:

Support of EXT:solrfal:

The following commands have been added:

  • solrfal:queue:delete
    Deletes items in the file index queue                                                           
  • solrfal:queue:reset-errors 
    Resets the items in the file index queue that are marked as errored and allows to re-index them 
  • solrfal:queue:get
    Shows items that are currently in the file index queue                                          
  • solrfal:queue:index
    Processes the index queue and index the items into the solr index.    


New command solr:index:verify

This command allows you to compare the current state of the Apache Solr Server and the TYPO3 System. It allows you to identify:   

  • Which items are missing in Solr?
  • Which items are missing in TYPO3?
  • Delete the missing items in Solr that are missing in TYPO3 and Re-Queue the missing items in TYPO3 that are missing in Solr with the Option **--fix**

Thanks to Jens Jacobsen and Ueberbit for the work on that feature!

New command solr:connection:flushall

This command allows you to flush all connections from the sys_registry table. This might be required if you want to remove all connections and re-initialize them with solr:connection:update then.
    
Thanks to Patrick Daxboeck and Bibus for sponsoring the work on that feature.

EXT:solrdebugtools 1.1

The addon EXT:solrdebugtools can be used to visually debug the scores of a document. With this addon you can see, which document has which impact on the score.

With version 1.1.0 we only provide small changes:

  • The score information is only visible when you are logged in as admin in the backend
  • EXT:solrdebugtools can be installed with TYPO3 9 LTS.


EXT:solrmlt 3

Solrmlt can now be used with TYPO3 9 LTS and uses the Solarium PHP-API as well to query the Apache Solr server.

More detailed information about EXT:solrmlt can be found on github.

Summary

EXT:solr and the addons are available now in the following version combination an can be installed in TYPO3 8 & 9:

  •  EXT:solr 9.0.1 (public)
  •  EXT:tika 4.0.0 (public)
  •  EXT:solrmlt 3.0.0 (public)
  •  EXT:solrfluidgrouping 2.0.0 (addon)
  •  EXT:solrfal 6.0.1 (addon)
  •  EXT:solrconsole 2.0.1 (addon)
  •  EXT:solrdebugtools 1.1.0 (addon)

Thanks

The work on EXT:solr and our addons is only possible with the financial support of our partners.

Therefore we want to thank all our sponsors that already have signed-up for an EB 2019:

  • Amedick & Sommer Neue Medien GmbH
  • Arndtteunissen GmbH
  • b13 GmbH
  • bgm business websolutions GmbH & Co KG
  • BIBUS AG Group
  • Bitmotion GmbH
  • CS2 AG
  • datamints GmbH
  • Fachhochschule für öffentliche Verwaltung NRW
  • Gernot Leitgab
  • Getdesigned GmbH
  • Hirsch & Wölfl GmbH
  • ITK Rheinland
  • Kassenärztliche Vereinigung Bayerns (KZVB)
  • LOUISE INTERNET GmbH
  • medien.de mde GmbH
  • mehrwert intermediale kommunikation GmbH
  • mellowmessage GmbH
  • plan2net GmbH
  • punkt.de GmbH
  • ressourcenmangel dresden GmbH
  • Stadt Villingen-Schwenningen
  • TOUMORO
  • Ueberbit Gmbh
  • XIMA MEDIA GmbH

Especially i want to thank our premium partners in 2019:

  • jweiland.net
  • sitegeist media solutions GmbH

Thanks also to Patrick Daxboeck from www.bibus.ch to involving us into the upgrade process to TYPO3 9 LTS. This helps us to improve EXT:solr and other components in a bigger live environment.

Try it out?

If you want to try EXT:solr & EXT:tika together with TYPO3 9.5.4, you can use our ddev setup, based on the introduction package.

In a few minutes you can setup your TYPO3 and Apache Solr development environment.

How to get started?

Are you excited about TYPO3 & Apache Solr and want to learn, how you can build a fast and powerful search on top of TYPO3?

We provide several appointments for a training, where you will learn, what Apache Solr is and how you can use it in your TYPO3 website.

Our next training dates are:

How to support us?

The work on EXT:solr and all the add-on's is financed with our partner program. With this partner program (EB) you:

  • Support the development of all open source components and the addon's
  • Support our support in the slack channel
  • Have access to several private add-on's (solrfal,solrdebugtools,solrconsole,solrfluidgrouping)

If you want to support us by becoming a partner please click here or call +49 (0)69 - 2475218 0


Über den Autor

Timo Hund

Timo Hund works as a senior developer at dkd Internet Service GmbH. Over the last year he collected a lot of experiences with Apache Solr and Lucene in several IT projects.

Mehr Beiträge von diesem Autor

Kommentar schreiben

* Diese Felder sind Pflichtfelder