da55499db0
fix: reject clients that do not send proper payload ( #18701 )
2023-12-22 01:26:17 -08:00
109a9e3f35
skip ILM expired objects from healing ( #18569 )
2023-12-01 07:56:24 -08:00
8317557f70
decom: Fix listing quorum to be equal to deletion quorum ( #18476 )
...
With an odd number of drives per erasure set setup, the write/quorum is
the half + 1; however the decommissioning listing will still list those
objects and does not consider those as stale.
Fix it by using (N+1)/2 formula.
Co-authored-by: Anis Elleuch <anis@min.io >
2023-11-17 21:09:09 -08:00
9a877734b2
Fix various poolmeta races ( #18230 )
...
There is a fundamental race condition in `newErasureServerPools`, where setObjectLayer is
called before the poolMeta has been loaded/populated.
We add a placeholder value to this field but disable all saving of the value, so we don't risk
overwriting the value on disk. Once the value has been loaded or created, it is replaced with
the proper value, which will also be saved.
Also fixes various accesses of `poolMeta` that were done without locks.
We make the `poolMeta.IsSuspended` return false, even if we shouldn't risk out-of-bounds
reads anymore.
2023-10-12 15:30:42 -07:00
9dc29d7687
Avoid ILM expiry on deleted versions that are yet to replicate ( #18175 )
...
Fixes #18167
2023-10-06 06:55:15 -06:00
22d2dbc4e6
decom: Fix infinite retry when the decom is canceled ( #18143 )
...
Also, use rand.Float64() since it is thread-safe; otherwise go race
will complain.
2023-09-30 00:02:29 -07:00
9947c01c8e
feat: SSE-KMS use uuid instead of read all data to md5. ( #17958 )
2023-09-18 10:00:54 -07:00
fa6d082bfd
reduce all major allocations in replication path ( #18032 )
...
- remove targetClient for passing around via replicationObjectInfo{}
- remove cloing to object info unnecessarily
- remove objectInfo from replicationObjectInfo{} (only require necessary fields)
2023-09-16 02:28:06 -07:00
1c99fb106c
Update to minio/pkg/v2 ( #17967 )
2023-09-04 12:57:37 -07:00
bddd53d6d2
fix: retry listing in decommissioning if it fails perpetually ( #17682 )
2023-07-19 13:09:37 -07:00
dfd7cca0d2
fix: allow cancel of decom only when its in progress ( #17607 )
2023-07-10 07:55:38 -07:00
d2f5c3621f
fix: add additional decommission traces for ILM expired content ( #17522 )
...
current decommission traces were missing for
- Skipped ILM expired versions
- Skipped single DELETE marked version
- A success or failure in decommissioning DELETE marker
- allow additional info to be shared in DecomStatus() API
2023-06-27 11:59:40 -07:00
eefa047974
fix: keep decommission in a go-routine ( #17496 )
...
This was removed by mistake in #17491
2023-06-23 12:29:32 -07:00
d315d012a4
decom: during multiple pool decom preserve current pool status ( #17491 )
...
removal of completed pools must retain pool status of other
pools in draining, to resume any remaining draining operations.
2023-06-23 07:44:18 -07:00
9af6c6ceef
under rebalance look for expired versions v/s remaining versions ( #17482 )
...
A continuation of PR #17479 for rebalance behavior must
also match the decommission behavior.
Fixes bug where rebalance would ignore rebalancing object
versions after one of the version returned "ObjectNotFound"
2023-06-21 13:23:20 -07:00
ccc5801112
always look for expired versions v/s remaining versions ( #17479 )
...
while decommissioning it can so happen that the non-current
versions are all expired but there is a DEL marker as the
latest version.
For such objects, we should not decommission them instead
calculate the remaining versions and if the remaining versions
is one and that version is a DEL marker consider such
an object not to be scheduled for decommissioning.
2023-06-21 08:49:28 -07:00
15911c85f6
safely ignore out of band deletions while decommissioning ( #17473 )
2023-06-20 08:31:42 -07:00
5a1612fe32
Bump up madmin-go and pkg deps ( #17469 )
2023-06-19 17:53:08 -07:00
38342b1df5
decom: Parallelize decommissining ( #17364 )
2023-06-07 14:27:51 -07:00
1436858347
log: Add a log when saving pool.bin fails ( #17338 )
...
Co-authored-by: Anis Elleuch <anis@min.io >
2023-06-04 14:20:21 -07:00
b210ea79bc
do not save MTime in newMultipartUpload() to avoid side-affects ( #17340 )
2023-06-02 14:38:09 -07:00
9b5829c16e
avoid decommissioning DEL markers with single versions ( #17274 )
2023-05-25 09:18:49 -07:00
3e128c116e
Add lifecycle event source to audit log tags ( #17248 )
2023-05-22 15:28:56 -07:00
06557fe8be
allow decommissioned pools to be removed while others are finishing ( #17221 )
2023-05-16 16:00:57 -07:00
aaf1abc993
simplify HardLimitReader by using LimitReader for internal usage ( #17218 )
2023-05-16 13:14:37 -07:00
e07c2ab868
Use hash.NewLimitReader for internal multipart calls ( #17191 )
2023-05-12 11:19:08 -07:00
883c98e26f
fix: remove objects when there are skipped versions due to ILM in decom ( #17198 )
2023-05-12 10:37:38 -07:00
3637aad36e
do not count ILM expired objects and other skipped objects ( #17184 )
2023-05-11 13:35:16 -07:00
b92cdea578
fix: start using pkg/workers to spawn parallel workers ( #17170 )
2023-05-09 16:37:31 -07:00
e7cac8acef
Add tags to auditLogLifecycle ( #17081 )
2023-04-26 17:49:00 -07:00
6825bd7e75
fix: inlined objects don't need to honor long locks ( #17039 )
2023-04-17 12:16:37 -07:00
d1e775313d
support decommissioning of tiered objects ( #16751 )
2023-03-16 07:48:05 -07:00
b984bf8d1a
allow expiration of all versions during Listing() ( #16757 )
2023-03-09 15:15:30 -08:00
31188e9327
add parallel workers in batch replication ( #16609 )
2023-02-13 12:07:58 -08:00
d67a846ec4
allow restarting of decommissioning if completed, failed or canceld ( #16464 )
2023-01-24 07:07:59 -08:00
095fc0561d
feat: allow decom of multiple pools ( #16416 )
2023-01-16 21:36:34 +05:30
475a88b555
fix: error out if an object is found after a full decom ( #16277 )
2023-01-12 05:52:51 +05:30
b882310e2b
avoid locks for internal and invalid buckets in MakeBucket() ( #16302 )
2022-12-23 07:46:00 -08:00
a30cfdd88f
Bump up madmin-go to v2 ( #16162 )
2022-12-06 13:46:50 -08:00
5a8df7efb3
re-implement StorageInfo to be a peer call ( #16155 )
2022-12-01 14:31:35 -08:00
fe8eed963e
fix: wrapped error will not equal in decommissioning ( #16113 )
2022-11-24 08:00:42 -08:00
853c4de75a
allow changing endpoints in distributed setups ( #16071 )
2022-11-16 07:59:10 -08:00
1f3db03bf0
allow changing argument for path for SNSD setup ( #16013 )
2022-11-07 00:11:58 -08:00
4523da6543
feat: introduce pool-level rebalance ( #15483 )
2022-10-25 12:36:57 -07:00
fc6c794972
Audit dangling object removal ( #15933 )
2022-10-24 11:35:07 -07:00
ac85c2af76
lifecycle: refactor rules filtering and tagging support ( #15914 )
2022-10-21 10:46:53 -07:00
928feb0889
remove unused debug param from evalActionFromLifecycle ( #15813 )
2022-10-07 10:24:12 -07:00
158d0e26a2
decom: Ignore object/version error during deletion ( #15806 )
2022-10-06 09:41:58 -07:00
a9f1ad7924
Add extended checksum support ( #15433 )
2022-08-29 16:57:16 -07:00
433b6fa8fe
upgrade golang-lint to the latest ( #15600 )
2022-08-26 12:52:29 -07:00
Harshavardhana