Commit Graph

436 Commits

Author SHA1 Message Date
Harshavardhana dd9ed85e22 implement support for FTP/SFTP server (#16952) 2023-04-15 07:34:02 -07:00
Anis Eleuch 91b6fe1af3 trace: Bootstrap to show the correct source line number (#16989) 2023-04-06 17:51:53 -07:00
Krishnan Parthasarathi 31fba6f434 Save bootstrap trace events in a circular buffer (#16823) 2023-03-17 16:01:03 -07:00
Harshavardhana 0c1f8b4e0f add user-agent for all minio.Client usage (#16619) 2023-02-14 13:19:30 -08:00
Harshavardhana 71f02adfca Revert "Print golang http errors in MinIO log format (#16465)"
This reverts commit 1fd7946dce.
2023-02-09 09:27:27 +05:30
Krishnan Parthasarathi 990fc415f7 Ensure safety of transitionState at startup (#16563) 2023-02-07 23:11:42 -08:00
Harshavardhana 747d475e76 initialize subsystems that are not dependent on buckets first (#16559) 2023-02-07 12:46:47 -08:00
Anis Elleuch 095b518802 Show a better error msg when internal data encryption key is incorrect (#16549) 2023-02-07 05:22:54 -08:00
Anis Elleuch 1fd7946dce Print golang http errors in MinIO log format (#16465) 2023-01-26 22:46:16 +05:30
Harshavardhana 54b561898f fix: anonymize the x-amz-id-2 value from hostname (#16478) 2023-01-25 10:25:36 -08:00
Shireesh Anjal 5a9f7516d6 Add monthly license update job (#16391) 2023-01-17 05:08:15 +05:30
Anis Elleuch 2146ed4033 xl: Quit early when EC config is incorrect (#16390)
Co-authored-by: Anis Elleuch <anis@min.io>
2023-01-09 23:07:45 -08:00
Harshavardhana e0086c1be7 reduce startup delays on kubernetes (#16356) 2023-01-05 02:32:43 -08:00
Harshavardhana 1cd8e1d8b6 remove the startup jitter before locks() (#16340) 2023-01-02 01:40:09 -08:00
Anis Elleuch acc9c033ed debug: Add X-Amz-Request-ID to lock/unlock calls (#16309) 2022-12-23 19:49:07 -08:00
Anis Elleuch 34167c51d5 trace: Add bootstrap tracing events (#16286) 2022-12-21 15:52:29 -08:00
Harshavardhana 5a218f38a1 allow retries for transaction lock on startup (#16273) 2022-12-19 22:00:00 -08:00
Anis Elleuch e57e946206 Do not save credentials in config.json (#16275) 2022-12-19 12:27:06 -08:00
Harshavardhana 80fc3a8a52 use newDynamicTimeoutWithOpts() when appropriate (#16266) 2022-12-15 13:11:37 -08:00
Klaus Post 988a2e8fed Faster startup of large distributed systems with latency (#16259) 2022-12-15 08:31:21 -08:00
Anis Elleuch 939c0100a6 log: Do not interpret verbs in object names in console output (#16233) 2022-12-13 08:27:40 -08:00
Aditya Manthramurthy 2d60bf8c50 Refactor HTTP transports (#16222) 2022-12-12 20:31:21 -08:00
Harshavardhana 37e20f6ef2 feat: allow listening specific addrs for API port (#16223) 2022-12-12 18:48:46 -08:00
Harshavardhana 853c4de75a allow changing endpoints in distributed setups (#16071) 2022-11-16 07:59:10 -08:00
Poorna d6bc141bd1 feat: Add support for site level resync (#15753) 2022-11-14 07:16:40 -08:00
Anis Elleuch 3b1a9b9fdf Use the same lock for the scanner and site replication healing (#15985) 2022-11-08 08:55:55 -08:00
Harshavardhana 9547b7d0e9 add deadlineConnections on remoteTransport (#16010) 2022-11-05 11:09:21 -07:00
Harshavardhana 6e4acf0504 add a message of removal for gateway and hide the command (#15965) 2022-10-28 14:11:20 -07:00
Harshavardhana 23b329b9df remove gateway completely (#15929) 2022-10-24 17:44:15 -07:00
Anis Elleuch 58d776daa0 Set CONSOLE_MINIO_SERVER to 127.0.0.1 by default (#15887) 2022-10-21 14:42:28 -07:00
Anis Elleuch de5070446d Deprecate --listeners flag (#15900) 2022-10-19 08:45:50 -07:00
Harshavardhana 6cb2f56395 Revert "Revert "tests: Add context cancelation (#15374)""
This reverts commit 564a0afae1.
2022-10-14 03:08:40 -07:00
Harshavardhana 2a13cc28f2 feat: implement support batch replication (#15554) 2022-10-05 23:00:43 -07:00
Anis Elleuch 86bb48792c non-blocking initialization of bucket target notifications (#15571) 2022-09-27 17:23:28 -07:00
Harshavardhana 9d6fddcfdf persist the non-default creds in config (#15711) 2022-09-21 16:14:47 -07:00
Klaus Post ff12080ff5 Remove deprecated io/ioutil (#15707) 2022-09-19 11:05:16 -07:00
Aditya Manthramurthy afbb63a197 Factor out external event notification funcs (#15574)
This change moves external event notification functionality into
`event-notification.go`. This simplifies notification related code.
2022-08-24 06:42:36 -07:00
Poorna 21fe14201f replication: centralize healthcheck for remote targets (#15516)
This PR moves health check from minio-go client to being
managed on the server.

Additionally integrating health check into site replication
2022-08-16 17:46:22 -07:00
Harshavardhana c7d535c648 init console after IAM init() (#15531)
fixes #15527
2022-08-13 12:54:41 -07:00
ebozduman b57e7321e7 Replaces 'disk'=>'drive' visible to end user (#15464) 2022-08-04 16:10:08 -07:00
Poorna 426c902b87 site replication: fix healing of bucket deletes. (#15377)
This PR changes the handling of bucket deletes for site 
replicated setups to hold on to deleted bucket state until 
it syncs to all the clusters participating in site replication.
2022-07-25 17:51:32 -07:00
Minio Trusted 564a0afae1 Revert "tests: Add context cancelation (#15374)"
This reverts commit 1e332f0eb1.

Reverting this as tests are failing randomly.
2022-07-21 13:58:56 -07:00
Klaus Post 1e332f0eb1 tests: Add context cancelation (#15374)
A huge number of goroutines would build up from various monitors

When creating test filesystems provide a context so they can shut down when no longer needed.
2022-07-21 11:52:18 -07:00
LHHDZ e68e76e143 fix: data race, which caused tests execution to fail (#15313) 2022-07-16 07:57:55 -07:00
Harshavardhana b4eb74f5ff allow custom speedtest bucket (#15271)
this allows for specifying existing buckets with

- object replication enabled
- object encryption enabled
- object versioning enabled
- object locking enabled
2022-07-12 10:12:47 -07:00
Harshavardhana 1a40c7c27c use signature-v2 for 'object perf' tests to avoid CPU using sha256 (#15151)
It is observed in a local 8 drive system the CPU seems to be
bottlenecked at

```
(pprof) top
Showing nodes accounting for 1385.31s, 88.47% of 1565.88s total
Dropped 1304 nodes (cum <= 7.83s)
Showing top 10 nodes out of 159
      flat  flat%   sum%        cum   cum%
      724s 46.24% 46.24%       724s 46.24%  crypto/sha256.block
   219.04s 13.99% 60.22%    226.63s 14.47%  syscall.Syscall
   158.04s 10.09% 70.32%    158.04s 10.09%  runtime.memmove
   127.58s  8.15% 78.46%    127.58s  8.15%  crypto/md5.block
    58.67s  3.75% 82.21%     58.67s  3.75%  github.com/minio/highwayhash.updateAVX2
    40.07s  2.56% 84.77%     40.07s  2.56%  runtime.epollwait
    33.76s  2.16% 86.93%     33.76s  2.16%  github.com/klauspost/reedsolomon._galMulAVX512Parallel84
     8.88s  0.57% 87.49%     11.56s  0.74%  runtime.step
     7.84s   0.5% 87.99%      7.84s   0.5%  runtime.memclrNoHeapPointers
     7.43s  0.47% 88.47%     22.18s  1.42%  runtime.pcvalue
```

Bonus changes:

- re-use transport for bucket replication clients, also site replication clients.
- use 32KiB buffer for all read and writes at transport layer seems to help
  TLS read connections.
- Do not have 'MaxConnsPerHost' this is problematic to be used with net/http
  connection pooling 'MaxIdleConnsPerHost' is enough.
2022-06-22 16:28:25 -07:00
Andreas Auernhammer cd7a0a9757 fips: simplify TLS configuration (#15127)
This commit simplifies the TLS configuration.
It inlines the FIPS / non-FIPS code.

Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2022-06-21 07:54:48 -07:00
Poorna 55ee94bed0 initialize site replication subsys after loading metadata (#15099) 2022-06-16 19:00:35 -07:00
Anis Elleuch 0d00f3a55b kms: initialize after cli parsing (#15076)
KMS depends on the --certs-dir flag. 

Ensure KMS is initialized after loading the flag.
2022-06-13 13:06:13 -07:00
Harshavardhana af1944f28d support reading systemctl config automatically on baremetal setups (#15066)
this allows for customers to use `mc admin service restart`
directly even when performing RPM, DEB upgrades. Upon such 'restart'
after upgrade MinIO will re-read the /etc/default/minio for any
newer environment variables.

As long as `MINIO_CONFIG_ENV_FILE=/etc/default/minio` is set, this
is honored.
2022-06-10 09:59:15 -07:00