社区 › 网站技术讨论区: Website Technology › Magento2.4实战百万商品之一：安装篇

suwubee 发表于 2020-8-27 23:17:12

Magento2.4实战百万商品之一：安装篇

本帖最后由 suwubee 于 2020-8-27 23:24 编辑


[*]前言：Magento2.4.0已经发布，为了重新认识这个越来越复杂的系统，决定从安装开始，边摸索边进行。


[*]硬件: 母鸡为Windows的R5-3600，64G, 512*2 NVME，通过hyper-v虚拟出3台内网互通的虚拟机，且全新安装Centos7.8.2003 Minimal


[*]架构: 1台机器配置Web、Mysql、Redis，作为主服务器)，第2台机器配置ElasticSearch，第3台机器配置Varnish
            这三台机器的配置、内网IP和hostname分别为            6核/16G，192.168.1.1，server1            1核/4G，192.168.1.2，server2            1核/4G，192.168.1.3，server3

[*]首先安装2号机器的ElasticSearch：
升级系统，安装Javayum -y updateyum -y install java-1.8.0-openjdkjava-1.8.0-openjdk-devel
配置Javacat <<EOF | sudo tee /etc/profile.d/java8.sh
export JAVA_HOME=/usr/lib/jvm/jre-openjdk
export PATH=\$PATH:\$JAVA_HOME/bin
export CLASSPATH=.:\$JAVA_HOME/jre/lib:\$JAVA_HOME/lib:\$JAVA_HOME/lib/tools.jar
EOF
source /etc/profile.d/java8.sh
配置elasticsearch安装源cat <<EOF | sudo tee /etc/yum.repos.d/elasticsearch.repo

name=Elasticsearch repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/oss-7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
EOF
安装elasticsearch
[*]yum -y install elasticsearch-oss
修改配置允许远程访问/etc/elasticsearch/elasticsearch.yml
修改为如下
network.host: 0.0.0.0
cluster.initial_master_nodes: ["node-1"]
启动elasticsearch并开机自启动systemctl enable --now elasticsearch
查看是否正常启动systemctl status elasticsearch
如果看到Active: active (running) 则是正常启动了。此外需要开放防火墙的9200端口。由于我这边是内网，所以直接把firewalld关闭，禁用自启动了。systemctl disable --now firewalld
再次测试是否运行，也可以在server1和server3上测试是否允许外部访问# curl http://127.0.0.1:9200
{
"name" : "server2",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "G2Jh9gIeSfK8D_D8hEYfOg",
"version" : {
    "number" : "7.9.0",
    "build_flavor" : "oss",
    "build_type" : "rpm",
    "build_hash" : "a479a2a7fce0389512d6a9361301708b92dff667",
    "build_date" : "2020-08-11T21:36:48.204330Z",
    "build_snapshot" : false,
    "lucene_version" : "8.6.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}


[*]安装3号机器varnish6.4.0：
一开始在jemalloc上折腾了很久，后来发现直接用epel源就顺利解决了yum install -y autoconf epel-release pygpgme yum-utils
配置安装源cat <<EOF | sudo tee /etc/yum.repos.d/varnishcache_varnish64.repo

name=varnishcache_varnish64
baseurl=https://packagecloud.io/varnishcache/varnish64/el/7/$basearch
repo_gpgcheck=1
gpgcheck=0
enabled=1
gpgkey=https://packagecloud.io/varnishcache/varnish64/gpgkey
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt
metadata_expire=300


name=varnishcache_varnish64-source
baseurl=https://packagecloud.io/varnishcache/varnish64/el/7/SRPMS
repo_gpgcheck=1
gpgcheck=0
enabled=1
gpgkey=https://packagecloud.io/varnishcache/varnish64/gpgkey
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt
metadata_expire=300
EOF
清除缓存，安装Varnish6.4.0sudo yum -q makecache -y --disablerepo='*' --enablerepo='varnishcache_varnish64'
yum -y install varnish

启动并开启自启动systemctl enable --now varnish
检查varnish状态systemctl status varnish
看到Active: active (running)则说明启动成功同样的由于内网，3号机器也关闭了防火墙systemctl disable --now firewalld<div align="left"></div>
后续在Magento2.4安装好之后，后台Store - Configuration - System - Full Page Cache配置Varnish参数，下载vcl配置文件。然后替换/etc/varnish/default.vcl，并重启即可。varnish可以通过命令varnishstat来查看运行状态

[*]安装1号机器Magento2.4：
为了这篇文章写得方便，这里使用目前比较流行的宝塔面板，如果是我自己使用的话，首选cPanel+Cloudlinux，其次用Vestacp。安装宝塔yum install -y wget && wget -O install.sh http://download.bt.cn/install/install_6.0.sh && sh install.sh
安装后选择安装软件、Apache 2.4.46
MySQL 8.0.20
PHP 7.4.9
Redis 6.0.6
【重要】安装完成后安装PHP扩展，ionCube、fileinfo、opcache、redis、intl、xsl，并且修改PHP的memory_limit为1536M，另外务必别忘了解除禁用函数putenv,proc_open,openlog,syslog,pcntl_signal。

然后进入宝塔面板，创建网站和数据库，并删除网站目录下的所有文件，避免安装时候文件重复无法覆盖。提前在https://marketplace.magento.com/customer/accessKeys/创建一个应用，获取Public Key和Private Key，使用composer安装，当然也可以登录这个页面下载源码https://magento.com/tech-resources/download
在composer的时候需要使用到unzip，因此预先安装好，并且通过ssh进入到网站主目录yum install -y unzipcd /www/wwwroot/demo.com
这里有三个办法安装Magento2.4主程序【推荐】方法一：单文件安装
在主目录生成composer.json文件cat <<EOF | sudo tee /www/wwwroot/demo.com/composer.json
{
    "name": "magento/project-community-edition",
    "description": "eCommerce Platform for Growth (Community Edition)",
    "type": "project",
    "license": [
      "OSL-3.0",
      "AFL-3.0"
    ],
    "config": {
      "preferred-install": "dist",
      "sort-packages": true
    },
    "require": {
      "magento/product-community-edition": "2.4.0",
      "magento/composer-root-update-plugin": "~1.0"
    },
    "require-dev": {
      "allure-framework/allure-phpunit": "~1.2.0",
      "dealerdirect/phpcodesniffer-composer-installer": "^0.5.0",
      "friendsofphp/php-cs-fixer": "~2.16.0",
      "lusitanian/oauth": "~0.8.10",
      "magento/magento-coding-standard": "*",
      "magento/magento2-functional-testing-framework": "^3.0",
      "pdepend/pdepend": "~2.7.1",
      "phpcompatibility/php-compatibility": "^9.3",
      "phpmd/phpmd": "^2.8.0",
      "phpstan/phpstan": ">=0.12.3 <=0.12.23",
      "phpunit/phpunit": "^9",
      "sebastian/phpcpd": "~5.0.0",
      "squizlabs/php_codesniffer": "~3.5.4"
    },
    "conflict": {
      "gene/bluefoot": "*"
    },
    "autoload": {
      "psr-4": {
            "Magento\\\\Framework\\\\": "lib/internal/Magento/Framework/",
            "Magento\\\\Setup\\\\": "setup/src/Magento/Setup/",
            "Magento\\\\": "app/code/Magento/",
            "Zend\\\\Mvc\\\\Controller\\\\": "setup/src/Zend/Mvc/Controller/"
      },
      "psr-0": {
            "": [
                "app/code/",
                "generated/code/"
            ]
      },
      "files": [
            "app/etc/NonComposerComponentRegistration.php"
      ],
      "exclude-from-classmap": [
            "**/dev/**",
            "**/update/**",
            "**/Test/**"
      ]
    },
    "autoload-dev": {
      "psr-4": {
            "Magento\\\\Sniffs\\\\": "dev/tests/static/framework/Magento/Sniffs/",
            "Magento\\\\Tools\\\\": "dev/tools/Magento/Tools/",
            "Magento\\\\Tools\\\\Sanity\\\\": "dev/build/publication/sanity/Magento/Tools/Sanity/",
            "Magento\\\\TestFramework\\\\Inspection\\\\": "dev/tests/static/framework/Magento/TestFramework/Inspection/",
            "Magento\\\\TestFramework\\\\Utility\\\\": "dev/tests/static/framework/Magento/TestFramework/Utility/",
            "Magento\\\\PhpStan\\\\": "dev/tests/static/framework/Magento/PhpStan/"
      }
    },
    "version": "2.4.0",
    "minimum-stability": "stable",
    "repositories": [
      {
            "type": "composer",
            "url": "https://repo.magento.com/"
      }
    ],
    "extra": {
      "magento-force": "override"
    }
}
EOF
方法二：composer获取源码composer create-project --repository-url=https://repo.magento.com/ magento/project-community-edition=2.4.0
#提示username输入Public Key，password输入Private Key，是否创建auth.json，输入Y回车。这个时候安装的数据都在/www/wwwroot/demo.com/project-community-edition目录下，我们需要转移文件到网站根目录，用命令行迁移容易丢失.htaccess等文件，建议用文件管理器迁移。
【推荐】方法三：上传源码在官网https://magento.com/tech-resources/download下载源码，并且解压到主目录。
上述三种办法任选其一，随后在主目录执行composer install
composer安装完成后执行magento安装，Magento2.4.0已取消网页安装，将下列命令行关于数据库信息和域名信息修改后填入，请注意ElasticSearch服务器是必需的bin/magento setup:install --base-url=http://www.demo.com/ \
--db-host=localhost --db-name=demo_com --db-user=demo_com --db-password=MmZ8asfdjk \
--admin-firstname=Magento --admin-lastname=User [email protected] \
--admin-user=admin --admin-password=demoadmin88 --language=en_US \
--currency=USD --timezone=America/Chicago --use-rewrites=1 \
--search-engine=elasticsearch7 --elasticsearch-host=server2 \
--elasticsearch-port=9200
执行完成后显示对应目录为/admin_31b6ig: Magento installation complete.
: Magento Admin URI: /admin_31b6ig
Nothing to import.
因为我们一直都是root用户执行命令，因此需要最后重置一下文件权限，此外Magento2.4.0强制开启了二次验证，我们这里关掉，随后刷新静态资源、索引和缓存chown -R www:www /www/wwwroot/demo.com
bin/magento module:disable Magento_TwoFactorAuth
bin/magento setup:upgrade
bin/magento setup:di:compile
bin/magento setup:static-content:deploy -f
bin/magento indexer:reindex
bin/magento cache:flush
如果遇到文件读写权限的问题，再执行一遍权限chown -R www:www /www/wwwroot/demo.com
安装完成后给Magento跑个分，不知道啥时候出的这个功能，需要注意这里要给mysql的root权限，或者直接改app/etc/env.php中的数据库用户为root，跑分文件位于setup/performance-toolkit/profiles/ce/目录下，我们选择一个中等配置进行测试，下面可以看到这个配置中测试生成了24000个简单产品，50000个订单。# bin/magento setup:performance:generate-fixtures /www/wwwroot/demo.com/setup/performance-toolkit/profiles/ce/medium.xml
Generating profile with following params:
|- Admin Users: 50
|- Websites: 3
|- Store Groups Count: 3
|- Store Views Count: 3
|- Categories: 300
|- Attribute Sets (Default): 3
|- Attribute Sets (Extra): 30
|- Simple products: 24000
|- Configurable products: 640
|- Product images: 1000, 3 per product
|- Customers: 2000
|- Cart Price Rules: 20
|- Catalog Price Rules: 20
|- Coupon Codes: 20
|- Orders: 50000
Config Changes...done in 00:00:00
Generating admin users...done in 00:00:02
Generating websites, stores and store views...done in 00:00:02
Generating categories...done in 00:00:11
Generating attribute sets...done in 00:00:10
Generating simple products...done in 00:01:55
Generating configurable EAV variations...done in 00:00:00
Generating bundle products...done in 00:00:00
Generating configurable products...done in 00:00:51
Generating images...done in 00:00:44
Generating customer groups...done in 00:00:00
Generating customers...done in 00:00:11
Generating cart price rules...done in 00:00:03
Generating catalog price rules...done in 00:00:03
Generating tax rates...done in 00:04:05
Generating tax rules...done in 00:00:00
Generating coupon codes...done in 00:00:00
Generating orders...done in 00:03:36
Indexers Mode Changes...done in 00:00:03
Design Config Grid index has been rebuilt successfully in 00:00:00
Customer Grid index has been rebuilt successfully in 00:00:01
Category Products index has been rebuilt successfully in 00:00:15
Product Categories index has been rebuilt successfully in 00:00:00
Catalog Rule Product index has been rebuilt successfully in 00:00:19
Product EAV index has been rebuilt successfully in 00:00:20
Stock index has been rebuilt successfully in 00:00:16
Inventory index has been rebuilt successfully in 00:00:00
Catalog Product Rule index has been rebuilt successfully in 00:00:00
Product Price index has been rebuilt successfully in 00:00:28
Catalog Search index has been rebuilt successfully in 00:03:55
Total execution time: 00:17:53
整体数据比较满意，毕竟4万产品和5万订单在这么短的时间内生成处理就已经相当不错了，而且这个服务器还只是入门性能。
跑完分后访问前后台还是有报错，没办法，再来执行一遍bin/magento setup:upgrade
bin/magento setup:di:compile
bin/magento setup:static-content:deploy -f
bin/magento indexer:reindex
bin/magento cache:flush
最后配置varnish，在后台依次进入Store - Configuration - System - Full Page Cache设置Backend host源IP，注意：我们这里是将Varnish分离出去了，因为实际情况中鉴于各种BUG以及对SSL兼容性的问题，最终我们将不使用varnish，本文仅作安装配置说明，后面在关于性能优化方面会具体为什么不用varnish以及varnish的替代方案。另外设置varnish为缓存后会存在前台导航菜单丢失的情况，最简单的办法是修改/vendor/magento/module-theme/view/frontend/layout/default.xml中66行，将ttl="3600"删除即可。配置Varnish后前台丢导航

到这里，整个环境安装完成。
Magento2.4.0首次登录后台显示
后台40000产品，非常流畅。
得益于ElasticSearch7.6，前台搜索也非常棒！
后续计划第二篇将通过一系列工具演练产品的采集和上传，第三篇会通过一些插件进而完善电商的各个运营环节，第四篇会注重优化前台页面的速度性能，后续待定...【我是原作者，转发公众号至广告中国，请勿转载。】

user 发表于 2020-8-28 08:42:32

4WSKU还是SPU,这么多产品上新索引大概多久啊,就怕后期慢

sead 发表于 2020-8-28 11:28:47

user 发表于 2020-8-28 08:42
4WSKU还是SPU,这么多产品上新索引大概多久啊,就怕后期慢

硬件到位就好，出了名的吃硬件:lol

suwubee 发表于 2020-8-28 14:33:17

user 发表于 2020-8-28 08:42
4WSKU还是SPU,这么多产品上新索引大概多久啊,就怕后期慢

跑分那一段都有时间，4w目前我刷新大概1-2分钟吧。后期会实战100w以上的数据。
Design Config Grid index has been rebuilt successfully in 00:00:00
Customer Grid index has been rebuilt successfully in 00:00:01
Category Products index has been rebuilt successfully in 00:00:15
Product Categories index has been rebuilt successfully in 00:00:00
Catalog Rule Product index has been rebuilt successfully in 00:00:19
Product EAV index has been rebuilt successfully in 00:00:20
Stock index has been rebuilt successfully in 00:00:16
Inventory index has been rebuilt successfully in 00:00:00
Catalog Product Rule index has been rebuilt successfully in 00:00:00
Product Price index has been rebuilt successfully in 00:00:28
Catalog Search index has been rebuilt successfully in 00:03:55

suwubee 发表于 2020-8-28 14:58:18

sead 发表于 2020-8-28 11:28
硬件到位就好，出了名的吃硬件

看架构吧，小型的不合适，大的产品多的也只能用跟这个了。这个母鸡我才300多一个月。

user 发表于 2020-8-28 21:53:14

suwubee 发表于 2020-8-28 14:33
跑分那一段都有时间，4w目前我刷新大概1-2分钟吧。后期会实战100w以上的数据。
...

谢谢反馈,我也上2.4版本试试

affempire 发表于 2020-8-31 11:01:04

期待后续佳作，这篇不错

knight5172 发表于 2021-1-13 17:54:30

学习路过

affiliateberry 发表于 2021-1-16 18:08:21

技术大牛

查看完整版本: Magento2.4实战百万商品之一：安装篇