Commit Graph

170 Commits

Author SHA1 Message Date
Harshavardhana e12ab486a2 avoid using os.Getenv for internal code, use env.Get() instead (#17688) 2023-07-20 07:52:49 -07:00
Harshavardhana 3e196fa7b3 fix: ILM newer noncurrent version limit must return correct versions (#17652)
objects/versions that are not expired via NewerNoncurrentVersions
must be properly returned to be applied under further ILM actions.

this would cause legitimately expired objects to be missed
from expiration.
2023-07-14 16:42:35 -07:00
Harshavardhana a566bcf613 treat 0-byte objects to honor same quorum as delete marker (#17633)
on unversioned buckets its possible that 0-byte objects
might lose quorum on flaky systems, allow them to be same
as DELETE markers. Since practically speak they have no
content.
2023-07-11 21:53:49 -07:00
Harshavardhana 82075e8e3a use strconv variants to improve on performance per 'op' (#17626)
```
BenchmarkItoa
BenchmarkItoa-8         	673628088	         1.946 ns/op	       0 B/op	       0 allocs/op
BenchmarkFormatInt
BenchmarkFormatInt-8    	592919769	         2.012 ns/op	       0 B/op	       0 allocs/op
BenchmarkSprint
BenchmarkSprint-8       	26149144	        49.06 ns/op	       2 B/op	       1 allocs/op
BenchmarkSprintBool
BenchmarkSprintBool-8   	26440180	        45.92 ns/op	       4 B/op	       1 allocs/op
BenchmarkFormatBool
BenchmarkFormatBool-8   	1000000000	         0.2558 ns/op	       0 B/op	       0 allocs/op
```
2023-07-11 07:46:58 -07:00
Harshavardhana 8522905d97 update helm linting workflow 2023-07-10 20:11:51 -07:00
Ian Martin 90c2129f44 Add helm chart linting to CI workflow (#17606) 2023-07-08 15:41:12 -07:00
Anis Eleuch 66bea3942a CI/CD to stop one node per pool in the two pools mint test (#17518)
This is to make sure that all S3 ops work when there is enough quorum
2023-07-07 00:10:13 -07:00
Harshavardhana 6b7c98bd0f make sure we pick up the right Go version in vulncheck (#17388) 2023-06-09 00:32:12 -07:00
Harshavardhana 75c6fc4f02 only allow decryption of etag for only sse-s3 (#17335) 2023-06-05 13:08:51 -07:00
Harshavardhana 4a425cbac1 cleanup scripts and apply shfmt (#17284) 2023-05-25 22:07:25 -07:00
Harshavardhana 876f51a708 remove minio-js from mint tests until next minio-js release 2023-05-17 09:09:54 -07:00
Harshavardhana 06557fe8be allow decommissioned pools to be removed while others are finishing (#17221) 2023-05-16 16:00:57 -07:00
Harshavardhana b53376a3a4 change directory objects to never create new versions (#17109) 2023-05-02 16:09:33 -07:00
Harshavardhana 7ae69accc0 allow root user to be disabled via config settings (#17089) 2023-04-28 12:24:14 -07:00
Klaus Post 62c3df0ca3 fix: directory listing on Go 1.20 windows (#16976) 2023-04-05 14:36:49 -07:00
Klaus Post ae011663e8 mint: Ignore teardown errors (#16979) 2023-04-05 11:10:24 -07:00
Harshavardhana f65cce4317 move mint tests to separate folders to not confuse GitHub (#16940) 2023-03-31 14:38:10 -07:00
Harshavardhana 13151cbb2b [testing] add mint runner test (#16868) 2023-03-29 11:38:43 -07:00
Aditya Manthramurthy c95ede35c1 Switch windows CI back to go 1.19.x (#16755) 2023-03-03 15:19:28 -08:00
Aditya Manthramurthy 7415e1aa56 Switch to go1.20 in CI (#16743) 2023-03-03 10:15:03 -08:00
Cesar Celis Hernandez 422c396d73 Removing old action that is no longer needed (#16550) 2023-02-07 07:06:29 +05:30
Cesar Celis Hernandez 8ca14e6267 Updating enterprise action (#16518) 2023-02-02 19:23:31 +05:30
Alex 1a17fc17bb GitHub Workflows security hardening (#15708) 2023-01-21 09:55:17 -08:00
Harshavardhana 75faef888e disable builds for go1.18 (#16332) 2022-12-30 11:37:07 -08:00
Harshavardhana dfe73629a3 fix: delete marker discrepancies via DeleteObject() API (#16195) 2022-12-08 18:15:16 -08:00
Anis Elleuch 52f4124678 Remove go1.18 from Github workflow tests (#16180) 2022-12-06 09:11:20 -08:00
Cesar Celis Hernandez 90e37a8745 Start PR on Enterprise when there is new MinIO Version (#16121) 2022-12-04 02:29:25 -08:00
yanggang d2c9a9e395 add windows port allot by "netsh dynamicport" (#15986) 2022-11-02 09:10:26 -07:00
Harshavardhana 07b6dce1a5 remove nancy, we rely on vulncheck from now on (#15893) 2022-10-18 10:45:44 -07:00
Minio Trusted 5c034e26bd fix: govulncheck must use go1.19.2 2022-10-06 12:42:59 -07:00
Harshavardhana cef0fb1434 remove build asset caching (#15807) 2022-10-06 11:36:07 -07:00
Harshavardhana b04c0697e1 validate correct ETag for the parts sent during CompleteMultipart (#15751) 2022-09-23 21:17:08 -07:00
Klaus Post 5c61c3ccdc Fix flaky TestGetObjectWithOutdatedDisks (#15687)
On occasion this test fails:

```
2022-09-12T17:22:44.6562737Z === RUN   TestGetObjectWithOutdatedDisks
2022-09-12T17:22:44.6563751Z     erasure-object_test.go:1214: Test 2: Expected data to have md5sum = `c946b71bb69c07daf25470742c967e7c`, found `7d16d23f07072af1a809707ba101ae07`
2
```

Theory: Both objects are written with the same timestamp due to lower timer resolution on Windows. This results in secondary resolution, which is deterministic, but random.

Solution: Instead of hacking in a wait we request the specific version we want. Should still keep the test relevant.

Bonus: Remote action dependency for vulncheck
2022-09-14 08:17:39 -07:00
Harshavardhana a71629d4dd use go1.19.1 for vulncheck (#15680) 2022-09-12 20:16:39 -07:00
Andreas Auernhammer d6132b854f add vulncheck action to CI (#15676)
Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2022-09-12 10:10:30 -07:00
Anis Elleuch bb855499e1 Update PR template to make it easy to request for doc changes (#15665)
A link can be clicked to open a new request in minio/docs if something
in the documentation needs to be changed.
2022-09-07 15:25:02 -07:00
Harshavardhana 433b6fa8fe upgrade golang-lint to the latest (#15600) 2022-08-26 12:52:29 -07:00
Harshavardhana d350b666ff feat: add idempotent delete marker support (#15521)
The bottom line is delete markers are a nuisance,
most applications are not version aware and this
has simply complicated the version management.

AWS S3 gave an unnecessary complication overhead
for customers, they need to now manage these
markers by applying ILM settings and clean
them up on a regular basis.

To make matters worse all these delete markers
get replicated as well in a replicated setup,
requiring two ILM settings on each site.

This PR is an attempt to address this inferior
implementation by deviating MinIO towards an
idempotent delete marker implementation i.e
MinIO will never create any more than single
consecutive delete markers.

This significantly reduces operational overhead
by making versioning more useful for real data.

This is an S3 spec deviation for pragmatic reasons.
2022-08-18 16:41:59 -07:00
Harshavardhana a6e0ec4e6f Add support converting non-inlined to inlined (#15444)
This is a feature to allow for inode compaction on
large clusters that use a lot of small files spread
across a large heirarchy.
2022-08-02 23:10:22 -07:00
Naveen d46cf50760 chore(deps): Included dependency review (#14958)
> Dependency Review GitHub Action in your repository to enforce dependency
> reviews on your pull requests.
> The action scans for vulnerable versions of dependencies introduced by package version
> changes in pull requests,
> and warns you about the associated security vulnerabilities.
> This gives you better visibility of what's changing in a pull request,
> and helps prevent vulnerabilities from being added to your repository.

https://docs.github.com/en/code-security/supply-chain-security/understanding-your-software-supply-chain/about-dependency-review#dependency-review-enforcement

Signed-off-by: Naveen <172697+naveensrinivasan@users.noreply.github.com>
2022-07-27 20:53:26 -07:00
Harshavardhana 4c6498d726 move all CI/CD to go1.18 (#15401) 2022-07-25 15:27:20 -07:00
Harshavardhana b6eb8dff64 Add decommission compression+encryption enabled tests (#15322)
update compression environment variables to follow
the expected sub-system style, however support fallback
mode.
2022-07-17 08:43:14 -07:00
Anis Elleuch 1cfa2e04bc Add a github workflow test for root disk detection (#15267)
Use losetup to create fake disks, start a MinIO cluster, umount 
one disk, and fails if the mount point directory will have format.json
recreated. It should fail because the mount point directory will belong
to the root disk after unmount.
2022-07-13 16:29:44 -07:00
Harshavardhana b311abed31 decom IAM, Bucket metadata properly (#15220)
Current code incorrectly passed the
config asset object name while decommissioning,
make sure that we pass the right object name
to be hashed on the newer set of pools.

This PR fixes situations after a successful
decommission, the users and policies might go
missing due to wrong hashed set.
2022-07-04 14:02:54 -07:00
Aditya Manthramurthy addfa35d93 Add FIPS build to CI and add README.fips.md (#15038) 2022-06-04 18:25:37 -07:00
Aditya Manthramurthy 4629abd5a2 Add tests for Access Management Plugin (#14909) 2022-05-12 15:24:19 -07:00
Aditya Manthramurthy 0e502899a8 Add support for multiple OpenID providers with role policies (#14223)
- When using multiple providers, claim-based providers are not allowed. All
providers must use role policies.

- Update markdown config to allow `details` HTML element
2022-04-28 18:27:09 -07:00
Harshavardhana 507f993075 attempt to real resolve when there is a quorum failure on reads (#14613) 2022-04-20 12:49:05 -07:00
Harshavardhana 2a6a40e93b enable go1.18.x builds (#14746) 2022-04-13 14:21:55 -07:00
Naveen 879de20edf Set permissions for GitHub actions (#14693)
- Included permissions for the action. https://github.com/ossf/scorecard/blob/main/docs/checks.md#token-permissions

https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs

[Keeping your GitHub Actions and workflows secure Part 1: Preventing pwn requests](https://securitylab.github.com/research/github-actions-preventing-pwn-requests/)

> Restrict the GitHub token permissions only to the required ones; this way, even if the attackers will succeed in compromising your workflow, they won’t be able to do much.

https://www.legitsecurity.com/blog/github-privilege-escalation-vulnerability

Signed-off-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>
2022-04-11 02:45:59 -07:00