Naguel

I'm Nahuel, and these are my work experiences, ideas and thoughts as a web developer working on the eCommerce industry.

Install Elasticsearch for Magento 2 on macOS

Install Elasticsearch for Magento 2 on macOS

Depending on the Magento version you have, or the Elasticsearch version you want, the process of getting this work locally can be easy or a nightmare I'll try to simplify here in this post.

If you already know what version to install, skip the following and jump straight to the installation instructions.

What Elasticsearch version do you need?

If we dive into the official Magento documentation and take a quick look at the different changes made to it across time, we can learn a few things.

Magento 2.3.4 documentation

Available at the magento/devdocs' GitHub repo, tag 2.3.4.

Magento 2.3.1 adds support for Elasticsearch 6.x and it is enabled by default. Magento still provides connectivity for Elasticsearch 2.x and 5.x but these must be enabled in order to use these versions.

Magento 2.3.5 documentation

Available at the magento/devdocs' GitHub repo, tag 2.3.5.

Magento 2.3.5 adds support for Elasticsearch 7.x.x (default) and 6.8.x. Both ES 2.x and 5.x are End of Life and are no longer supported in Magento.

Magento 2.4 documentation

Available at the magento/devdocs' GitHub repo, branch 2.4.1-develop.

You must install and configure Elasticsearch 7.6.x before upgrading to Magento 2.4.0.
Magento does not support Elasticsearch 2.x, 5.x, and 6.x.

Or, to summarize:

  • For Magento 2.3.1 to Magento 2.3.4 you need to install Elasticsearch 6.x.
  • For Magento 2.3.5 to Magento 2.3.n (anything before 2.4.0) you'll need to install Elasticsearch 7.x.x. It works also with 6.8.x, but version 7.x.x is easy to install.
  • For Magento 2.4.0 and up you need to install Elasticsearch 7.6.x. No support for previous versions.

If you are working with multiple Magento projects on different versions, I would suggest you to install Elasticsearch 6.x to support from Magento 2.3.1 to Magento 2.3.5, and everything before Magento 2.4.

If you have all projects on Magento 2.3.5 and up then install Elasticsearch 7.6.x.

Install Elasticsearch 6.x on macOS

You need Homebrew first, so please install it by doing the following:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"

You can check the official Homebrew page for more information about it and alternative installations options.

Then, we need to add a third-party repo to Homebrew and a few packages, including Elasticsearch itself.

brew tap elastic/tap; brew cask install homebrew/cask-versions/adoptopenjdk8; brew install elasticsearch@6.8;

When you do that you'll get a lot of output on your terminal, but you need to only care for the installation path of the last elasticsearch@6.8 package.

➜  brew install elasticsearch@6.8;
==> Downloading https://homebrew.bintray.com/bottles/elasticsearch%406.8-6.8.8.catalina.bottle.tar.gz
Already downloaded: /Users/nahuelsanchez/Library/Caches/Homebrew/downloads/01d0782011cbecdb3b1125469ee2ed60ef07926a8f36752c64d1b8c2b763736d--elasticsearch@6.8-6.8.8.catalina.bottle.tar.gz
==> Pouring elasticsearch@6.8-6.8.8.catalina.bottle.tar.gz
==> /usr/local/Cellar/elasticsearch@6.8/6.8.8/bin/elasticsearch-keystore create
==> Caveats
Data: /usr/local/var/lib/elasticsearch/
Logs: /usr/local/var/log/elasticsearch/elasticsearch_nahuelsanchez.log
Plugins: /usr/local/var/elasticsearch/plugins/
Config: /usr/local/etc/elasticsearch/

elasticsearch@6.8 is keg-only, which means it was not symlinked into /usr/local,
because this is an alternate version of another formula.

If you need to have elasticsearch@6.8 first in your PATH run:
  echo 'export PATH="/usr/local/opt/elasticsearch@6.8/bin:$PATH"' >> ~/.zshrc
  
To have launchd start elasticsearch@6.8 now and restart at login:
  brew services start elasticsearch@6.8
Or, if you don't want/need a background service you can just run:
  elasticsearch
==> Summary
🍺  /usr/local/Cellar/elasticsearch@6.8/6.8.8: 136 files, 103.4MB

The path below "Summary" is the one we need to mind, where mine is /usr/local/Cellar/elasticsearch@6.8/6.8.8.

Go to that folder and install a few Elasticsearch plugins.

bin/elasticsearch-plugin install analysis-phonetic bin/elasticsearch-plugin install analysis-icu; bin/elasticsearch-plugin install analysis-smartcn;

Now, because only God knows why, we need to fix something on this freshly installed Elasticsearch package, otherwise it won't start.

Standing again into that installation path, cd into the libexec/config folders, and edit the jvm.options file.

Comment the line it says 8:-Xloggc:logs/gc.log by adding a # at the beginning of it, and below add 8:-Xloggc:/tmp/logs_gc.log.

This is how the file might ended up looking

You are basically done now.

Start Elasticsearch by doing brew services start elasticsearch@6.8.

➜  brew services start elasticsearch@6.8
==> Successfully started `elasticsearch@6.8` (label: homebrew.mxcl.elasticsearch@6.8)

To check if everything is good, you can do a cURL to the Elasticsearch instance running.

➜  curl "http://localhost:9200/_nodes/settings?pretty=true"
{
  "_nodes" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "cluster_name" : "elasticsearch_brew",
  "nodes" : {
    "ArBWW_c3QRunhfQn460sxQ" : {
      "name" : "ArBWW_c",
      "transport_address" : "127.0.0.1:9300",
      "host" : "127.0.0.1",
      "ip" : "127.0.0.1",
      "version" : "6.8.8",
      "build_flavor" : "oss",
      "build_type" : "tar",
      "build_hash" : "2f4c224",
      "roles" : [
        "master",
        "data",
        "ingest"
      ],
      "settings" : {
        "client" : {
          "type" : "node"
        },
        "cluster" : {
          "name" : "elasticsearch_brew"
        },
        "http" : {
          "type" : {
            "default" : "netty4"
          }
        },
        "node" : {
          "name" : "ArBWW_c"
        },
        "path" : {
          "data" : [
            "/usr/local/var/lib/elasticsearch"
          ],
          "logs" : "/usr/local/var/log/elasticsearch",
          "home" : "/usr/local/Cellar/elasticsearch@6.8/6.8.8/libexec"
        },
        "transport" : {
          "type" : {
            "default" : "netty4"
          }
        }
      }
    }
  }
}

That host and port on that cURL command (which by default is localhost:9200) is the host and port you need to use for configuring Elasticsearch in Magento. If it doesn't work, try the same cURL call but with 127.0.0.1:9200.

Remember to brew services stop elasticsearch@6.8 when you are done working so it's not running forever in your computer.

Install Elasticsearch 7.x on macOS

You need Homebrew first, so please install it by doing the following:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"

You can check the official Homebrew page for more information about it and alternative installations options.

Then, we need to add a third-party repo to Homebrew and the Elasticsearch package itself.

brew tap elastic/tap; brew install elastic/tap/elasticsearch-full;

When you do that you'll get a lot of output on your terminal, but you need to only care for the installation path of the elastic/tap/elasticsearch-full package, which is located just at the end below "Summary".

For example, mine says /usr/local/Cellar/elasticsearch-full/7.8.0.

Go to that folder and install a few Elasticsearch plugins.

bin/elasticsearch-plugin install analysis-phonetic bin/elasticsearch-plugin install analysis-icu; bin/elasticsearch-plugin install analysis-smartcn;

That's all, basically.

Start Elasticsearch by doing brew services start elastic/tap/elasticsearch-full.

To check if everything is good, you can do a cURL to the Elasticsearch instance by running curl "http://localhost:9200/_nodes/settings?pretty=true" which will output a big JSON.

That host and port on that cURL command (which by default is localhost:9200) is the host and port you need to use for configuring Elasticsearch in Magento. If it doesn't work, try the same cURL call but with 127.0.0.1:9200.

Remember to brew services stop elastic/tap/elasticsearch-full when you are done working so it's not running forever in your computer.

Can I have both installed at the same time?

No... well, I didn't try it but I think you can't. I would avoid such experiment.

What you can do for sure is uninstall one and install the other on demand.

If you do that you need to uninstall the Elasticsearch plugins you install it and install them again so you ended up with the right plugins version compatible with the Elasticsearch version currently installed.

Anything else?

Reindex Magento before trying to load the store.

Magento virtual themes or why theme changes don't show up in the front end

Magento virtual themes or why theme changes don't show up in the front end

You have a Magento 2 theme created (at least you can see it in your IDE and we're going to assume there's no typo in the theme.xml nor in the registration.php), you can also see it in the Admin, and finally you have it configured for your store.

Then you go to the front end and you see a Frankenstein's monster: the CSS is okay but the Layout XML is not being taken into consideration. The front end kind of knows which theme to load, but it's loading it only partially.

What it's going on? How do we fix it?

One of the possibilities is that the theme is set as Virtual instead of Physical in the database, causing behaviors such as the described above.

Magento 2 happen to have 3 types of themes, of which I have found very little information in the official documentation (just a minor mention in the Troubleshooting section of the 2.0 "Apply and configure a storefront theme" page but gone in the 2.2 version of it) but a convincing explanation in this StackExchange question.

We have "Physical themes" which are the well-known themes such as Blank and Luma, second we have "Virtual themes" which appears to be themes that extend a Physical theme but doesn't exist in the files (nothing in app/design), and finally we have the "Staging themes" which I have no idea what they are nor how they behave (sorry).

These 3 different types of themes are defined in the Interface Magento\Framework\View\Design\ThemeInterface as 0, 1, and 2 for Physical, Virtual and Staging, respectively.

If you go to your database (using the Terminal, PhpMyAdmin or an app like TablePlus) and take a look at the theme table, chances are that you'll find something like this:

If you encounter a 1 in the type column for your theme, there's your problem. Change it to 0 and everything will be fixed.

How the theme became Virtual without anybody setting it anywhere?

You're probably wondering how that 1 ended up there without anybody setting it (at least you're sure you didn't as you just find out about all this).

As described, Virtual themes are themes that doesn't exist in the code, meaning there are no files for that theme (how Magento expect us to use this type of theme is for me unknown).

If you're using a version control system such as Git maybe (maybe) you ended up for a moment in a branch without the theme files and somehow loaded the front end or ran a bin/magento command. For a moment only, Magento saw in the database that a theme is set and active but found no files in the code for such theme, so it changed it to Virtual.

About how hard it is to find a front end developer capable of work with Magento

About how hard it is to find a front end developer capable of work with Magento

Is it hard to find someone willing (and able?) to work with Magento. And I'm not talking about finding an already "senior" Magento front end developer with n years of experience with that eCommerce platform and/or working in that specific field already.

No. What I mean is that it is difficult to find someone who is already a front end developer but never worked with Magento. Will he be able to? Will he understand and like the platform? Is he willing to despite what he says during the interview?

I already wrote an article about how to interview a front end developer, and I'm not sure if I strengthened enough the idea of it being a kind of a bet.

How to interview a front end developer and not die in the process
Being an interviewer it’s not an easy task as the candidate’s chances of getting the job depends not only on the candidate’s technical skills but also on the ability of the interviewer to rule on the truth of those said skills in a 60 minutes talk.

In this scenario we're betting twice as we not only need to discover if the person being interviewed possesses the skills to be a good developer but also if he or she can perform with the specific platform.

That fuzzy line between back end and front end within Magento

I encountered a lot of different "profiles" while interviewing. While the spectrum is really broad let me summarize this into two groups (nevertheless, not sure about the names I'm giving to them, but bear with me):

  • Web designers, or people focused only on the HTML and CSS (maybe jQuery?) part of the coding process. People whose main task is to convert a given PSD file into "a web".
  • Developers using the "hard tools" for coding complex logical solutions with, and not limited to, JavaScript and all its modern flavours.

The first group is excellent in that PSD to xHTML conversion process but they sort of depend on somebody to add the functionality to their creations, and the later group fails when the QA team compares what they delivered with the original design files.

So, someone in between, right? Right. But why?, because of that fuzzy line between back end and front end within Magento. :)

It depends on how the company you work for is structured, and how the roles for back end and front end are implemented there, but under my experience I can tell that none of these groups that represents the two extremes of the wide range of developers I interview can work with Magento, or at least they can't on their own as they will always lack what the other groups knows... and we need both on the front end side of a Magento project for it to go live.

Again, under my experience, while working with people that falls into this two categories I either faced the problem of things looking good but not working as expected, or things that do what they're are supposed to do but with so many differences when I open the designs (and if you are thinking "it's only colors", think responsive web design).

How we find that "in between"?

Spoiler alert: I don't know for sure, but I can tell you what I'm doing right now and hopefully it will help you achieve the same as I'm looking for if you're conduction interviews.

I recently had a meeting with somebody from the recruiting team at the company I work for in order to polish the "mechanism" to identify people too attached to the mentioned groups in order to dismiss them at the beginning of the process, and there are a lot of hints in the candidate's CV that you can use.

Exhibit A is this imaginary resume that includes skills such as HTML, CSS, SASS and jQuery, which is excellent because we need those skills. Following we find a lot of background experience building corporate websites only, but never using a CMS as there is no mention for WordPress, Joomla, Drupal, or similar.

Finally, the work experience is a mix of marketing and design, mostly a transition from the later to a developer. The court rules guilty for being too much of a member of the web designers group.

A second example is this also imaginary resume that includes all the fancy words I already used to, such as Angular, AngularJS, React, VUE, Node... you name it... and a background experience that mostly includes working on the back end side of some sites.

I don't have to tell you to which group this imaginary candidate belongs to.

So, again, someone in between. Find that resume that have a mix of the trending topic's keywords for the first and second group, and you got yourself a feasible Magento front end developer. Or work the other way around: burn the resumes that falls into the extremes and what lays in the middle is the people you should interview next.

No group is better than the other

Don't get me wrong... or let me clarify.

I do not think one group is better than the other one, and if you belong to the first one there's no obligation to learn what the other group knows, neither the other way around.

Let me put another short perspective into this: if the company you work for (or the company you applied for, or your company if you happens to have one) really separates this two groups into really two separates job positions, and if you only have to worry about the PSD to xHTML thing because somebody else is handling the functionality... good for you, and go for it, as again this depends on the scenario you're standing.

That's why I mentioned something about the way a company is structured before, because in some places you might not need to worry about the "in between" as it doesn't apply.

Happy hunting.