Commit Graph

337 Commits

Author SHA1 Message Date
Harshavardhana 23b329b9df remove gateway completely (#15929) 2022-10-24 17:44:15 -07:00
Poorna e4e90b53c1 fix: delete-marker replication check properly (#15923) 2022-10-21 14:45:06 -07:00
Anis Elleuch ac85c2af76 lifecycle: refactor rules filtering and tagging support (#15914) 2022-10-21 10:46:53 -07:00
Aditya Manthramurthy 2d16e74f38 Add LDAP IDP Configuration APIs (#15840) 2022-10-19 11:00:10 -07:00
Klaus Post bd3dfad8b9 Add concurrent Snowball extraction + options (#15836) 2022-10-18 13:50:21 -07:00
Aditya Manthramurthy 85fc7cea97 Pass role ARN for OIDC providers to console (#15862) 2022-10-15 12:57:03 -07:00
Harshavardhana 328d660106 support CRC32 Checksums on single drive setup (#15873) 2022-10-15 11:58:47 -07:00
Harshavardhana 59e33b3b21 validate setBucketTarget properly as per BucketExists() call (#15860) 2022-10-13 17:46:49 -07:00
Poorna 0e3c92c027 attempt delete marker replication after object is replicated (#15857)
Ensure delete marker replication success, especially since the
recent optimizations to heal on HEAD, LIST and GET can force
replication attempts on delete marker before underlying object
version could have synced.
2022-10-13 17:45:23 -07:00
hellivan b0b573052a fix: unrecognized openid config parameters (#15847) 2022-10-12 12:19:44 -07:00
Javier Adriel 2939000342 Add metrics, version and apis handlers (#15839) 2022-10-12 12:08:03 -07:00
Anis Elleuch afd4279cd8 lock tests: Initialize different DRWMutex for each lock (#15833) 2022-10-10 15:14:32 -07:00
Harshavardhana 3c4ef4338f marshal retention XML in expected format (#15821) 2022-10-08 13:11:00 -07:00
Aditya Manthramurthy 64cf887b28 use LDAP config from minio/pkg to share with console (#15810) 2022-10-07 22:12:36 -07:00
Krishnan Parthasarathi 6d6a731d6d Handle overlapping and conflicting ILM rules (#15812) 2022-10-07 14:36:23 -07:00
Poorna b6b26dba87 fix: GetObjectRetention to parse in ISO8601 time format (#15809) 2022-10-06 13:53:56 -07:00
Harshavardhana 2a13cc28f2 feat: implement support batch replication (#15554) 2022-10-05 23:00:43 -07:00
Javier Adriel 3307aa1260 Implement KMS handlers (#15737) 2022-10-04 10:05:09 -07:00
Harshavardhana f696a221af allow tagging policy condition for GetObject (#15777) 2022-10-02 12:29:29 -07:00
Anis Elleuch 6287e8c571 fix: race when accessing REST TCP dial values (#15770) 2022-09-29 09:27:58 -07:00
Anis Elleuch f69a98ce49 fix: loading Audit kafka configuration loading (#15766) 2022-09-29 08:35:08 -07:00
Daryl White d44f3526dc Update links to documentation site (#15750) 2022-09-28 21:28:45 -07:00
Harshavardhana 41b633f5ea support tagging based policy conditions (#15763) 2022-09-28 11:25:46 -07:00
Anis Elleuch 86bb48792c non-blocking initialization of bucket target notifications (#15571) 2022-09-27 17:23:28 -07:00
Harshavardhana 94dbb4a427 fix: generalize SC config and also skip healing sub-sys under SD (#15757) 2022-09-26 09:04:54 -07:00
Anis Elleuch 048a46ec2a Add RPC tcp timeout/errs and AVG duration to prometheus (#15747) 2022-09-26 09:04:26 -07:00
Harshavardhana 877bd95fa3 remove unused package internal/smart (#15758) 2022-09-25 08:44:53 -07:00
Poorna 8ea6fb368d Add auto configuration of replication workers (#15636) 2022-09-24 16:20:28 -07:00
Poorna 5fd5ddea23 relax retention date validation on replication target (#15752) 2022-09-23 21:19:03 -07:00
Anis Elleuch 20c89ebbb3 freeze before exit when _MINIO_DEBUG_NO_EXIT is defined (#15709)
this is to ensure keep k8s pods running, when they reach a "crashloop" stage
2022-09-22 11:57:27 -07:00
Krishnan Parthasarathi 6f56ba80b3 lifecycle: Assign unique id to rules with empty id (#15731) 2022-09-22 10:51:54 -07:00
Klaus Post ff12080ff5 Remove deprecated io/ioutil (#15707) 2022-09-19 11:05:16 -07:00
Javier Adriel 0b6175b742 Implement KMS methods for keys, policies and identities (#15673) 2022-09-19 11:04:40 -07:00
Klaus Post c22f3ca7a8 fix: S3 Select CSV -> JSON with variable field count (#15677)
When there are fewer fields than expected, output fewer fields.
2022-09-12 17:00:59 -07:00
Anis Elleuch 4a92134235 prometheus: track errors during REST read/write calls (#15678)
minio_inter_node_traffic_errors_total currently does not track
requests body write/read errors of internode REST communications.

This commit fixes this by wrapping resp.Body.
2022-09-12 12:40:51 -07:00
Poorna 6b9fd256e1 Persist in-memory replication stats to disk (#15594)
to avoid relying on scanner-calculated replication metrics.
This will improve the accuracy of the replication stats reported.

This PR also adds on to #15556 by handing replication
traffic that could not be queued by available workers to the 
MRF queue so that entries in `PENDING` status are healed faster.
2022-09-12 12:40:02 -07:00
Harshavardhana 228c6686f8 allow non-standards fallback for all http.TimeFormats (#15662)
fixes #15645
2022-09-07 07:24:54 -07:00
Krishnan Parthasarathi 5ce1448049 audit: account for response headers separately (#15610) 2022-09-01 12:51:04 -07:00
Klaus Post 8e4a45ec41 fix: encrypt checksums in metadata (#15620) 2022-08-31 08:13:23 -07:00
Abirdcfly d4e0f13bb3 chore: remove duplicate word in comments (#15607)
Signed-off-by: Abirdcfly <fp544037857@gmail.com>

Signed-off-by: Abirdcfly <fp544037857@gmail.com>
2022-08-30 08:26:43 -07:00
Anis Elleuch 1f28a3bb80 Avoid messages from go test output (#15601)
A lot of warning messages are printed in CI/CD failures generated by go
test. Avoid that by requiring at least Error level for logging when
doing go test.
2022-08-30 08:23:40 -07:00
Klaus Post a9f1ad7924 Add extended checksum support (#15433) 2022-08-29 16:57:16 -07:00
Harshavardhana 97376f6e8f improve performance for inlined data (#15603)
inlined data often is bigger than the allowed
O_DIRECT alignment, so potentially we can write
'xl.meta' without O_DSYNC instead we can rely on
O_DIRECT + fdatasync() instead.

This PR allows O_DIRECT on inlined data that
would gain the benefits of performing O_DIRECT,
eventually performing an fdatasync() at the end.

Performance boost can be observed here for small
objects < 128KiB. The performance boost is mainly
seen on HDD, and marginal on NVMe setups.
2022-08-29 11:19:29 -07:00
Harshavardhana 433b6fa8fe upgrade golang-lint to the latest (#15600) 2022-08-26 12:52:29 -07:00
Krishnan Parthasarathi 99fbfe2421 Add concurrency to healing objects on a fresh disk (#15575) 2022-08-25 13:07:15 -07:00
Aditya Manthramurthy 18dffb26e7 Allow querying a single target in config get API (#15587) 2022-08-25 00:17:05 -07:00
Aditya Manthramurthy c54c13831a fix: error types in internal config package (#15559) 2022-08-20 15:11:49 -07:00
Harshavardhana ae4ee95d25 change default lock retry interval to 50ms (#15560)
competing calls on the same object on versioned bucket
mutating calls on the same object may unexpected have
higher delays.

This can be reproduced with a replicated bucket
overwriting the same object writes, deletes repeatedly.

For longer locks like scanner keep the 1sec interval
2022-08-19 16:21:05 -07:00
Aditya Manthramurthy 21831b3fe2 fix: cleanup config KV parsing using madmin helpers (#15552) 2022-08-18 15:55:17 -07:00
Aditya Manthramurthy 701a82642b Ensure subnet+callhome configs are single sub-sys (#15542) 2022-08-16 17:46:46 -07:00